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This  study  was  undertaken  to  develop  explicit  definitions  of  preventable  drug- 
related  morbidity  (PDRM),  and  use  these  definitions  to  measure  PDRM  in  an  elderly 
managed  care  population.  The  Delphi  technique  was  used  with  a  geriatric  medicine 
expert  panel  to  come  to  consensus  on  49  explicit  definitions  of  PDRM  in  the  elderly. 
These  definitions  were  then  operationalized  using  ICD-9,  CPT  and  NDC  codes  that  were 
compatible  with  a  managed  care  organization's  database  of  patient  information.  There 
were  1 1,71 1  patients  eligible  for  the  study.  The  explicit  definitions  detected  at  least  one 
instance  of  preventable  drug-related  morbidity  in  8.2%  (966)  of  eligible  patients.  A  rate 
of  6.25  events  per  100  patient  years  was  detected. 

Evidence  for  validity  of  the  definitions  was  gathered  through  the  use  of  known 

risk  factors  and  health  care  resource  use.  The  presence  of  PDRM  was  significantly 

associated  with  the  number  of  prescriptions  a  patient  received,  the  number  of  diagnoses, 

the  number  of  prescribers,  and  patients  over  85.  Patients  with  a  preventable  drug-related 

xi 


morbidity  event  detected  also  used  significantly  more  health  care  resources  and  cost  more 
to  care  for  than  those  without  an  event.  The  resource  use  included  hospitalizations, 
emergency  room  visits,  physician  office  visits  and  medications. 

The  measure  of  preventable  drug-related  morbidity  was  used  as  part  of  a  quality 
assessment  and  improvement  exercise  to  demonstrate  its  utility  in  uncovering  quality 
problems  in  the  medication  use  system.  An  expert  panel  generated  four  cause-and-effect 
diagrams  that  could  be  used  to  structure  improvement  opportunities.  Thus,  the  measure 
was  useful  in  a  formative  system. 

This  study  is  important  in  its  development  of  explicit  definitions  for  PDRM  and 
linking  the  presence  of  the  pattern  of  care  with  the  corresponding  adverse  outcome. 
These  definitions  may  be  used  as  indicators  to  measure  and  monitor  the  performance  of 
the  medication  use  system.  Ultimately,  they  may  be  used  to  monitor  the  pharmaceutical 
care  of  individual  patients. 
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CHAPTER  1 
INTRODUCTION 

The  Need  for  the  Study 

Medications  are  the  most  common  treatment  prescribed  in  health  care.  The 
majority  of  patients  who  visit  the  doctor's  office  leave  with  a  prescription  (Schappert, 
1994).  However,  the  act  of  writing  a  prescription  does  not  assure  optimal  outcomes. 

Medication  use  requires  several  steps  to  achieve  the  desired  outcome.  These  steps 
appear  fragmented  in  today's  health  care  environment.  The  result  is  a  medication  use 
process  that  is  unsafe.  Problems  stem  from  systematic  flaws  in  the  process,  poor 
communication  among  participants  in  the  system,  the  lack  of  defined  therapeutic 
objectives,  and  inadequate  feedback  mechanisms  needed  for  systems  improvement 
(Grainger-Rousseau  et  al.,  1997;  Schaffetal.,  1991). 

A  process  of  poor  quality  will  often  produce  poor  outcomes.  In  terms  of 
medication  use,  a  poorly  functioning  process  leads  to  patients  receiving  sub-optimal  drug 
therapy.  Sub-optimal  drug  therapy  may  lead  to  patient  injury  (drug-related  morbidity)  or 
death  (drug-related  mortality).  Patient  injury  and  death  impact  both  patients  and  society. 
Studies  have  shown  that  patient  death  from  medication  use  is  an  important  issue  that  must 
be  addressed  (Institute  of  Medicine  Committee  on  Quality  Health  Care  in  America,  1999; 
Lazarou  et  al.,  1998;  Talley  &  Laventurier,  1974). 

Drug-related  morbidity  (DRM)  may  reduce  a  patient's  quality  of  life  and  ability  to 
function  in  society.  It  may  also  impact  the  patient  financially  through  direct  medical 
expenses  (i.e.,  co-pays  and  deductibles),  indirect  medical  expenses,  and  lost  productivity. 
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The  financial  impact  of  drug-related  morbidity  and  mortality  on  society  is 

enormous.  Johnson  and  Bootman  (1995)  estimated  that  drug-related  morbidity  and 

mortality  in  ambulatory  care  patients  cost  society  $76  billion  dollars  each  year.  This 

estimate  would  place  it  among  the  most  expensive  "disease  states"  and  most  important 

patient  safety  issues  in  health  care. 

Direct  health  care  resources  used  to  treat  drug-related  morbidity  are  substantial. 
Care  in  a  hospital  setting  is  expensive  and  is  often  used  to  treat  the  consequences  of  DRM 
(Bergman  &  Wiholm,  1981;  Bero  et  al.,  1991;  Bigby  et  al.,  1987;  Caranasos  et  al.,  1974; 
Hallas  et  al.,  1992;  Lakshmanan  et  al.,  1986;  Lindley  et  al.,  1992;  Nelson  &  Talbert, 
1996).  Studies  suggest  that  up  to  20%  of  all  hospital  admissions  are  drug-related 
(Bergman  et  al.,  1981;  Bero  et  al.,  1991;  Bigby  et  al.,  1987;  Caranasos  et  al.,  1974; 
Cunningham  et  al.,  1997;  Hallas  et  al.,  1992;  Lakshmanan  et  al.,  1986;  Lindley  et  al., 
1992;  Nelson  &  Talbert,  1996). 

Other  health  care  products  and  services  such  as  the  emergency  room,  physician 
office  visits,  long-term  care,  skilled  nursing  care,  and  medications  are  used  to  treat  DRM 
(Bootman  et  al.,  1997;  Dennehy  et  al.,  1996;  Hanlon  et  al.,  2000;  Johnson  &  Bootman, 
1995;  Rossiter  et  al.,  2000).  The  lost  opportunity  costs  for  the  health  care  system  are 
substantial. 

As  stated  above,  drug-related  morbidity  is  a  problem  in  the  medication  use  system 
that  reduces  the  quality  of  care  received  by  patients  and  increases  the  total  cost  of  health 
care.  Like  any  epidemic,  prevention  is  the  most  efficient  and  humane  strategy.  The  idea 
of  preventability  is  gaining  momentum  in  the  literature.  Research  suggests  that  around 
50%  of  DRM  events  are  preventable  (Bates  et  al.,  1995b;  Bates  et  al.,  1993;  Bigby  et  al., 
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1987;  Culler  et  al.,  1998;  Dubois  &  Brook,  1988;  Hallas  et  al.,  1990;  Nelson  &  Talbert, 

1 996).  Preventing  drug-related  morbidity  would  significantly  improve  the  safety  and 

quality  of  the  medical  care  system,  while  at  the  same  time  reducing  average  per-patient 

cost. 

Despite  the  increased  attention  preventability  has  received,  there  is  no  consensus 
in  the  literature  either  defining  or  measuring  its  prevalence  (Ross,  2001).  Because  there 
is  no  consistent  method  for  identifying  and  reporting  instances  of  preventable  drug- 
related  morbidity,  research  results  show  wide  variability  in  terms  of  the  rate  of  events  and 
their  preventability. 

Current  efforts  to  identify  and  evaluate  drug-related  events  usually  include 
intensive  surveillance  and  chart  review.  Implicit  review  methods  are  often  labor 
intensive,  expensive,  and  difficult  to  replicate  (Brennan,  2000;  Donabedian,  1978).  In 
addition,  many  published  studies  look  only  at  patients  in  the  hospital  setting  or  their 
reason  for  admission.  By  focusing  on  acute  care  settings,  many  problems  are  left 
undiscovered  (Karch  &  Lasagna,  1975).  More  effort  needs  to  be  directed  at  the  entire 
continuum  of  care,  specifically  ambulatory  care. 

Improved  methods  of  identifying  and  measuring  drug-related  morbidity  are 
needed  to  maximize  efforts  focused  on  preventing  these  events.  Emerging  computer 
technology  provides  a  new  opportunity  for  surveillance  and  identification  of  drug-related 
morbidity  (Maass  &  Cortezzo,  2000;  Raschke  et  al.,  1998).  The  use  of  computers  will 
facilitate  the  surveillance  of  large  populations  of  patients  across  care  sites.  This  may  be 
accomplished  by  developing  explicit  criteria  and  applying  them  to  computerized  patient 
information  as  an  indicator  of  potential  quality  problems.  Computer  screening  with 


4 
explicit  criteria  will  provide  more  complete  information,  speed  up  data  review,  and 

reduce  the  costs  associated  with  measurement.  Thus,  explicit  criteria  to  measure 

preventable  drug-related  morbidity  will  add  valuable  information  to  improve  the  quality 

of  medication  use  across  organizations. 

Problem  Statement 

The  medication  use  system  currently  in  place  is  fragmented  and  lacks  feedback 
necessary  for  improvement.  This  results  in  medication  use  that  is  not  optimal  and  may 
lead  to  patient  injury.  Drug-related  injury  is  a  problem  for  patients  and  the  health  care 
system.  Patients  may  experience  a  reduced  quality  of  life  or  even  death  as  a  result  of  the 
improper  use  of  medications.  Meanwhile,  the  health  care  system  is  spending  billions  of 
dollars  treating  patients  with  drug-related  morbidity  and  mortality  (Johnson  &  Bootman, 
1995;  Thomas  et  al.,  1999).  The  literature  suggests  that  about  50%  of  DRM  is 
preventable  (Bates  et  al.,  1995b;  Bates  et  al.,  1993;  Bigby  et  al.,  1987;  Culler  et  al.,  1998; 
Hallas  et  al.,  1990;  Nelson  &  Talbert,  1996).  Preventing  drug-related  morbidity  will 
result  in  substantial  savings  for  the  health  care  system  and  improve  the  quality  of  life  for 
patients. 

The  measurement  of  preventable  drug-related  morbidity  is  hindered  by  the  lack  of 
consensus  regarding  terminology  and  reliable  methodologies  (Ross,  2001).  The  health 
care  literature  currently  appears  to  support  expert  review  to  determine  rates  of  DRM. 
However,  implicit  review  by  experts  is  difficult  to  replicate  and  would  be  cost  prohibitive 
in  most  organizations.  Because  of  the  lack  of  widespread  measurement,  many  leaders  of 
health  care  organizations  may  be  unaware  that  preventable  drug-related  morbidity 
(PDRM)  is  a  problem.  Without  recognizing  PDRM  as  a  significant  safety  and  financial 
issue,  resources  will  not  be  directed  at  correcting  a  faulty  medication  use  system. 
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A  recent  report  by  the  Institute  of  Medicine  (1999)  recommends  the  development 

and  testing  of  indicators  to  track  medication  errors  and  to  develop  preventative  strategies 

based  on  the  results.  Explicit  definitions  for  PDRM  may  serve  this  purpose.  Explicit 

criteria  to  measure  PDRM  are  now  possible  with  advancing  computer  technology. 

MacKinnon  (1999)  has  defined  specific  scenarios  of  PDRM  that  could  be  used  as  explicit 

measures  of  PDRM  and  as  an  indicator  of  the  quality  of  the  medication  use  system.  This 

work  will  lead  to  more  efficient  methods  of  determining  the  prevalence  of  PDRM  and 

bring  much  needed  attention  to  the  issue. 

Study  Objectives 

There  are  four  objectives  in  this  study: 

1 .  To  develop  explicit  definitions  of  preventable  drug-related  morbidity, 

2.  To  identify  issues  in  operationalizing  the  definitions  of  PDRM  to  be  used  in  a 
computer  database, 

3.  To  measure  preventable  drug-related  morbidity  in  an  elderly  managed  care 
population,  and 

4.  To  use  the  measure  of  preventable  drug-related  morbidity  in  a  structured 
quality  assessment  and  improvement  framework  to  identify  quality  problems. 

Research  Questions 

To  accomplish  the  objectives  of  this  study,  several  research  questions  will  be 
investigated. 
Research  Question  1 

Can  the  definitions  developed  by  MacKinnon  ( 1 999)  be  face  and  content 
validated  in  a  different  physician  panel? 


Research  Question  2 

What  issues  are  involved  in  operationalizing  definitions  of  preventable  drug- 
related  morbidity  for  use  in  a  computer  database? 

Research  Question  3 

Can  PDRM  be  measured  in  an  existing  computer  database? 
Research  Question  4 

What  is  the  prevalence  of  PDRM  in  an  elderly  Medicare  population? 
Research  Question  5 

Are  there  risk  factors  that  are  more  common  in  patients  with  preventable-drug 

related  morbidity? 
Research  Question  6 

What  is  the  relationship  between  patients  with  PDRM  and  their  utilization  of 
health  care  resources? 
Research  Question  7 

Can  a  measure  of  PDRM  be  used  to  identify  breakdowns  in  quality  in  the 
medication  use  system? 


CHAPTER  2 
REVIEW  OF  LITERATURE  AND  FRAMEWORK 

Medication  Use 

Technological  advances  in  the  diagnosis  and  treatment  of  medical  conditions 
continue  at  unparalleled  levels  (Mootonen  et  al.,  2001).  The  information  and  resources 
available  to  health  care  practitioners  today  were  unheard  of  just  a  few  decades  ago.  One 
area  of  health  care  that  has  grown  exponentially  in  the  latter  half  of  the  twentieth  century, 
in  both  knowledge  and  available  options,  is  medication  therapy. 

Today,  medications  are  the  most  common  treatment  modality  used  in  the  United 
States  (Schappert,  1994).  Approximately  63  percent  of  all  physician  office  visits  result  in 
a  written  prescription  (Schappert,  1994).  Medications  are  an  important  component  of 
patient  wellness.  The  Joint  Commission  on  the  Accreditation  of  Healthcare 
Organizations  (JCAHO)  promotes  the  appropriate,  safe,  effective,  and  efficient  use  of 
medications  as  a  necessary  clinical  function  to  assure  positive  patient  outcomes  (Schaff  et 
al.,  1991). 

As  a  legitimate  therapeutic  alternative,  there  are  four  objectives  of  drug  therapy; 
(1)  cure  a  disease,  (2)  reduce  or  eliminate  symptoms  of  a  disease,  (3)  arrest  or  slow  the 
disease  process,  and  (4)  prevent  the  disease  or  symptoms  (Hepler  &  Strand,  1990). 
However,  the  act  of  prescribing  a  medication  does  not  assure  optimal  outcomes.  The 
medication  use  process  is  a  series  of  steps  that  must  operate  together  to  attain  the 
intended  outcome.  It  is  the  complexity  and  number  of  steps  in  the  medication  use  process 
that  increase  the  likelihood  of  an  adverse  outcome  (Leape,  1994). 
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Each  year,  an  estimated  1 .3  million  persons  are  injured  as  a  result  of  the  medical 

care  they  receive  (Cullen  et  al.,  1997).  Medications  have  been  implicated  as  the  leading 

cause  of  injuries  due  to  medical  care  (Brennan  et  al.,  1991;  Leape  et  al.,  1991).  Thus, 

despite  the  advances  in  technology,  education  and  training  of  health  care  practitioners, 

and  the  availability  of  newer  and  presumably  safer  drug  entities,  evidence  suggests  that 

patients  are  placed  at  risk  when  they  are  prescribed  medications  for  legitimate  therapeutic 

reasons  and  follow  advice  given  by  health  care  professionals. 

Impact  of  Drug-Related  Morbidity 

We  will  examine  three  issues  as  they  relate  to  drug-related  morbidity;  1)  how 
much  it  costs,  2)  the  impact  on  the  quality  of  care,  and  3)  the  impact  on  patient  access  to 
services. 
Cost 

Drug-related  morbidity  is  a  burden  for  society  in  terms  of  direct  health  care 
expenses.  Johnson  and  Bootman  (1995)  estimated  that  drug-related  morbidity  and 
mortality  cost  society  $76  billion  dollars  a  year  (1995  dollars).  This  amount  would  place 
drug-related  morbidity  and  mortality  among  the  most  expensive  disease  states. 

Several  studies  have  examined  the  impact  of  drug-related  morbidity  on  hospital 
admissions.  Studies  suggest  that  up  to  20%  of  all  hospital  admissions  are  drug-related 
(Bergman  &  Wiholm,  1981;  Bero  et  al.,  1991;  Bigby  et  al.,  1987;  Caranasos  et  al.,  1974; 
Cunningham  et  al.,  1997;  Hallas  et  al.,  1992;  Lakshmanan  et  al.,  1986;  Lindley  et  al., 
1 992;  Nelson  &  Talbert,  1996).  Hospital  costs  were  expected  to  reach  $5 1 8  billion  in  the 
year  2000  (Burner  &  Waldo,  1995).  Thus,  drug-related  hospital  admissions  could  cost 
$  1 03  billion,  or  approximately  7%  of  estimated  total  health  care  expenditures  in  2000 
(Burner  &  Waldo,  1995). 
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Drug-related  morbidity  occurring  in  hospitalized  patients  affects  both  cost  and 

quality.  Studies  have  demonstrated  that  patients  who  experience  an  adverse  drug  event 

(ADE)  in  the  hospital  cost  significantly  more  to  care  for  and  had  a  longer  length  of  stay. 

(Bates  et  al.,  1997;  Classen  et  al.,  1999;  Geraci  et  al.,  1999;  Pearson  et  al.,  1994; 

Schimmel,  1964;  Weingart  et  al.,  2000). 

Drug-related  morbidity  can  increase  the  number  of  emergency  room  visits, 
unscheduled  physician  office  visits,  nursing  home  admissions,  and  the  cost  of 
medications  a  patient  is  taking  (Bootman  et  al.,  1997;  Dennehy  et  al.,  1996;  Hanlon  et  al., 
2000;  Johnson  &  Bootman,  1995;  Rossiter  et  al.,  2000).  When  added  together,  the  total 
direct  heath  care  expenditures  to  treat  drug-related  morbidity  should  demand  attention. 
Quality 

Drug-related  deaths  may  be  the  most  alarming  quality  issue  in  health  care  today. 
The  1 999  Institute  of  Medicine  report  estimates  that  approximately  98,000  patients  die 
each  year  from  preventable  medical  errors  (Institute  of  Medicine  Committee  on  Quality 
Health  Care  in  America,  1999).  This  report  generated  debate  on  the  issue,  expanding 
beyond  the  medical  community.  Politicians  have  pledged  additional  dollars  to  study  and 
reduce  these  events.  However,  what  is  new  to  the  politicians  is  not  new  to  the  health  care 
system. 

In  1974,  Talley  and  Laventurier  estimated  that  up  to  140,000  deaths  each  year 
were  caused  by  adverse  drug  reactions.  Lazarou  et  al.  (1998)  performed  a  meta-analysis 
of  past  adverse  drug  reaction  studies  and  estimated  that  4.6%  of  recorded  deaths  in  the 
United  States  were  due  to  adverse  drug  reactions.  This  result  would  place  adverse  drug 
reactions  between  the  4th  to  6th  leading  cause  of  death  in  this  country.  These  figures 
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warrant  investigation  into  medication  use  and  methods  for  reducing  drug-related  injury 

and  death. 

Access 

Many  persons  in  the  United  States  have  limited  access  to  health  care  (Culler  et  al., 

1998).  This  limited  access  may  be  due  to  inadequate  health  insurance,  the  high  cost  of 

seeking  care  without  insurance,  or  the  lack  of  physical  facilities  or  health  care 

professionals  within  reasonable  travelling  distance. 

Drug-related  morbidity  is  increased  when  access  to  medical  care  is  limited. 

Culler  etal.  (1998)  state 

access  to  timely  and  effective  ambulatory  medical  care  may  reduce  the 
risk  of  hospitalization  for  selected  medical  conditions  either  by  preventing 
the  onset  of  disease,  by  controlling  an  acute  episodic  illness  or  condition, 
or  by  managing  a  chronic  disease  or  condition  in  such  a  way  as  to  prevent 
progression  of  the  disease,  (p.  804) 

Patients  who  do  not  seek  care  when  they  have  a  medical  indication  will  not  receive  the 
necessary  treatment  and  monitoring  to  alleviate  symptoms  or  cure  their  disease. 
Eventually,  the  patient's  condition  may  deteriorate  and  require  additional  health  care 
services. 

Access  can  be  further  impacted  by  a  patient's  attitude  towards  past  care.  If 
patients  are  dissatisfied  with  the  care  they  receive,  they  may  be  less  willing  to  access  the 
health  care  system  in  the  future  (Gandhi  et  al.,  2000). 

Preventability  of  Drug-Related  Morbidity 

As  with  any  medical  problem,  prevention  is  the  most  humane  strategy.  Formerly, 
the  preventability  of  adverse  events  due  to  medication  use  went  largely  unrecognized.  In 
1955,  Barr  stated  that  adverse  drug  reactions  were  the  "price  we  pay"  for  using  these 
chemical  agents.  Moser  (1956)  added  to  this  sentiment  by  stating  that  adverse  drug 
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reactions  were  a  "disease  of  medical  progress."  In  essence,  they  were  saying  that  we 
must  accept  the  risk  because  the  benefits  outweigh  any  harm.  While  these  statements 
were  made  in  the  1950's,  they  seem  to  reflect  the  thought  on  the  subject  for  many  years 
since  then.  This  prevailing  thought  is  illustrated  in  a  recent  article  by  Lazarou  et  al. 
(1998)  where  the  authors  seem  to  lament  the  perceived  lack  of  preventability  of  adverse 
drug  reactions. 

The  idea  of  preventability  had  not  received  a  lot  of  attention  prior  to  1970. 
Melmon  (1971)  was  one  of  the  early  authors  that  addressed  preventability.  Melmon 
(1971)  stated  that  70-80%  of  drug  reactions  were  predictable.  If  these  events  were 
predictable,  prevention  is  possible.  Melmon  (1971)  further  debunked  the  idea  that 
adverse  effects  were  a  "disease  of  medical  progress"  by  stating  these  events  were 
preventable  "without  compromise  of  the  therapeutic  benefits  of  the  drug"  (p.  1366). 
Melmon  (1971)  went  on  to  recommend  possible  causes  of  adverse  drug  reactions  and 
what  might  be  done  to  reduce  their  incidence  and  impact. 

Studies  now  indicate  that  around  50%  of  drug-related  morbidity  are  preventable 
(Bates  et  al.,  1995b;  Bates  et  al.,  1993;  Bigby  et  al.,  1987;  Culler  et  al.,  1998;  Dubois  & 
Brook,  1988;  Hallas  et  al.,  1990;  Hallas  et  al.,  1992;  Johnson  &  Bootman,  1995; 
Lakshmanan  et  al.,  1986;  Nelson  &  Talbert,  1996).  If  50%  of  drug-related  morbidity  can 
be  prevented,  significant  gains  in  the  quality  of  care  and  reductions  in  costs  are  possible. 
Preventing  half  of  Johnson  and  Bootman's  (1995)  cost-of-illness  estimate  would  yield 
savings  of  $38  billion  each  year.  The  impact  of  prevention  cannot  be  understated. 
However,  in  order  to  prevent  these  events,  it  is  necessary  to  understand  quality 
assessment  and  improvement  techniques  and  the  system  of  medication  use. 
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Quality  Assessment  and  Improvement 

Quality  assessment  and  improvement  programs  have  been  popular  in  health  care 
since  the  late  1980's.  These  programs  take  many  different  names  such  as  continuous 
quality  improvement  (CQI),  total  quality  management  (TQM),  performance  improvement 
(PI),  or  quality  improvement  (QI).  However,  their  basic  premise  is  the  same:  to  look  at 
the  system  of  care  in  place,  measure  the  current  level  of  function,  identify  and  implement 
improvement  opportunities,  and  re-measure  the  system  to  determine  the  impact  of  the 
changes.  One  framework  commonly  used  in  health  care  to  organize  the  process  is 
referred  to  by  the  acronym  FOCUS-PDCA.  The  Columbia/HCA  Corporation  (Nashville, 
Tennessee)  developed  the  FOCUS-PDCA  process  as  an  extension  of  the  PDCA  cycle 
(Gerard  &  Arnold,  1996).  The  following  steps  are  proposed  as  a  systematic  process  for 
quality  assessment  and  improvement: 

Find  a  process  to  improve. 

Organize  a  team  that  understands  the  process. 

Clarify  current  knowledge  of  the  system. 

Understand  the  causes  of  variation. 

Select  the  improvement  or  intervention. 

Plan  the  improvement. 

Do  implement  the  improvement. 

Check  the  improvement  by  gathering  and  analyzing  data. 

Act  on  the  data  to  hold  the  gain  or  continue  improving. 
Within  each  step  of  the  FOCUS-PDCA  process,  there  are  "tools"  available  to 
uncover  quality  problems  and  guide  the  identification  of  improvement  opportunities. 
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Some  of  these  tools  include  the  flow  chart,  run  charts,  control  charts,  Pareto  charts  and 

root  cause  analysis. 

The  theory  of  quality  assessment  and  improvement  (and  the  FOCUS-PDCA 
process)  fits  well  with  the  idea  of  preventing  drug-related  morbidity.  By  using  quality 
assessment  and  improvement  techniques,  flaws  in  the  medication  use  process,  which 
cause  PDPvM,  are  uncovered  and  changed  with  lasting  results.  This  will  require  a 
measure  of  PDRM  that  can  serve  as  a  valid  indicator  of  improvement  opportunities. 

An  indicator  is  a  tool  used  to  monitor  and  evaluate  the  quality  of  important 
governance,  management,  clinical,  and  support  functions  that  affect  patient  outcomes 
(Angaran,  1991).  Indicators  are  used  as  screens  to  detect  potential  problems  in  the 
quality  of  care.  Indicators  selected  should  tap  key  areas  of  the  medication  use  system. 
Many  quality  assessment  and  improvement  programs  operate  by  looking  at  the  structure, 
process,  and  outcomes  of  our  systems  (Donabedian,  1978).  To  examine  medication  use 
and  why  indicators  of  quality  problems  may  be  beneficial,  a  review  of  structure,  process 
and  outcome  is  warranted. 
Structure.  Process  and  Outcome 

Structure  refers  to  the  material  and  social  instrumentalities  that  are  used  to 
provide  care  (Donabedian,  1978).  These  may  include  the  physical  environment,  the 
training  and  qualifications  of  the  personnel,  and  organizational  issues.  The  structure 
component  of  the  medication  use  system  has  received  a  lot  of  attention  in  past  years  due 
to  efforts  by  the  Joint  Commission  on  Accreditation  of  Healthcare  Organizations 
(JCAHO),  state  boards  of  pharmacy,  and  others. 

The  process  and  outcome  components  of  quality  are  currently  receiving  the 
majority  of  attention  in  health  care.  Process  refers  to  those  steps  or  procedures  used  to 
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carry  out  the  care  of  patients.  Processes  of  care  are  important  insomuch  as  they  lead  to 

the  desired  outcome  (Donabedian,  1978).  Outcomes  are  defined  as  the  change  in  health 

status  that  can  be  attributed  to  the  care  provided  (Donabedian,  1978). 

A  poor  medication  use  process  can  reduce  the  quality  of  care  by  placing  a  patient 
at  risk  for  additional  harm  (i.e.,  poor  outcomes).  This  risk  can  manifest  itself  as  further 
medical  conditions,  a  worsening  of  symptoms,  a  reduction  in  the  patient's  quality  of  life, 
or  even  death.  Thus,  it  is  important  to  identify  key  measures  (indicators)  of  the 
medication  use  process  that  may  reflect  quality  problems.  Once  key  indicators  are  in 
place,  then  tools  such  as  root  cause  analysis  are  used  to  determine  where  the  breakdowns 
occur. 
Root  Cause  Analysis 

Root  cause  analysis  has  been  used  as  a  quality  assessment  and  improvement  tool 
in  health  care  (Fernandes  et  al.,  1997;  JCAHO,  1996;  Shinn,  2000).  Root  cause  analysis 
(RCA)  is  a  method  that  uncovers  the  causes  of  quality  problems  and  the  relationship 
between  the  cause  and  the  net  effect  (JCAHO,  1996).  It  is  important  to  note  that  root 
cause  analysis  does  not  establish  the  magnitude  of  the  individual  causes  identified.  The 
results  of  a  root  cause  analysis  lack  both  time  and  quantitative  data.  Additional  quality 
assessment  and  improvement  tools  are  needed  to  determine  which  quality  issues  are  most 
important. 

A  root  cause  analysis  allows  participants  in  a  problem  solving  process  to  utilize 
information  gathered  to  reason  out  possible  causes  for  poor  quality  (Shinn,  2000).  From 
this  reasoning  process,  system  changes  are  identified  and  implemented.  Thus,  the 
medication  use  process  is  amenable  to  a  root  cause  analysis. 
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The  Medication  Use  System 

The  medication  use  system  has  not  changed  appreciably  in  many  years.  However, 
the  demands  on  the  system  have  changed  dramatically  due  to  an  increase  in  technology 
and  the  availability  of  treatment  options  (Mootonen  et  al.,  2001).  The  medication  use 
process  has  become  very  complex  and  fragmented  in  nature  (Leape  et  al.,  2000).  There 
are  many  opportunities  for  breakdowns  in  the  system  when  components  of  the  process 
operate  independently.  The  lack  of  communication  among  the  doctor,  patient, 
pharmacist,  and  others  involved  in  patient  care  further  reduces  the  quality  and  safety  of 
medication  use  (Bates,  2000). 

The  current  medication  use  model  for  prescription  medications  is  detailed  in 
Figure  2.1  (emphasis  added  at  the  end)  (Nadzam,  1998).  In  ambulatory  care,  this  process 
begins  with  a  physician  office  visit  that  culminates  in  a  prescription  being  written.  The 
patient  is  then  responsible  to  have  the  prescription  filled  at  a  pharmacy.  At  that  point,  the 
patient  is  responsible  to  take  the  medication  and  monitor  results. 


Prescribing 


Dispensing 


Administration 


Monitoring 


? 


Figure  2.1 :  Current  Medication  Use  Process  (Nadzam,  1998) 


As  suggested  in  the  medication  use  model,  no  consistent  feedback  mechanism 
exists  to  detect  and  correct  drug-related  problems.  The  assumption  appears  to  be  that 
once  the  medication  is  prescribed  correctly  and  dispensed  properly,  optimal  outcomes  are 
assured.  Thus,  the  last  step  in  the  process  is  truly  the  end  until  another  office  visit  or 
drug-related  morbidity  occurs. 
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Given  the  limitations  of  the  current  medication  use  process,  there  is  a  need  to 

redesign  the  system  to  foster  the  prevention  of  adverse  events.  Hepler  and  Grainger- 

Rousseau  (1995)  have  proposed  a  medication  use  system  that  links  the  steps  of  the 

process  and  includes  an  integrated  monitoring  plan  directed  at  detecting  and  resolving 

drug-related  problems  (See  Figure  2.2).  The  addition  of  an  explicit  feedback  mechanism 

changes  the  process  into  a  system. 

As  medication  use  is  viewed  as  a  system,  quality  assessment  and  improvement 
activities  can  have  their  greatest  impact.  The  outcomes  by  which  success  is  judged  will 
become  more  obvious  as  quality  assessment  and  improvement  become  the  standard.  A 
system  that  identifies  goals  at  the  beginning  of  therapy  will  set  in  place  a  monitoring 
strategy  with  indicators  designed  to  detect  success  or  failure.  This  monitoring  system 
will  include  the  detection  and  resolution  of  drug-related  problems  as  they  arise. 
Defining  and  Measuring  Drug-Related  Morbidity 

Patient  injury  as  a  consequence  of  drug  therapy  is  a  popular  topic  in  the  literature. 

Over  the  last  few  decades,  many  attempts  to  define  drug-related  events  and  assess  their 

preventability  have  been  made.  Despite  these  efforts,  no  clear  consensus  exists  and  there 

is  much  confusion  regarding  terminology  (Ross,  2001).  A  review  of  the  terminology 

follows. 

Adverse  Drug  Reactions 

The  most  widely  recognized  and  measured  adverse  event  involving  drug  therapy  is  an 

adverse  drug  reaction  (ADR).  In  1970,  the  World  Health  Organization  (WHO)  defined 

an  adverse  drug  reaction  as 

any  response  that  is  noxious,  unintended,  and  undesired  and  that  occurs  at 
doses  normally  used  in  man  for  prophylaxis,  diagnosis,  or  therapy,  (p. 
103) 
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Figure  2.2:  Medication  Use  System  (Hepler  &  Grainger-Rousseau,  1995,  p.  6) 
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Despite  its  wide  use  in  the  literature,  the  World  Health  Organization's  definition 

of  an  ADR  has  limitations.  There  are  differing  opinions  on  whether  the  definition 

includes  the  failure  to  accomplish  the  intended  purpose  (drug  therapy  failure).  Bergman 

and  Wilholm  (1981)  believe  the  failure  to  accomplish  the  goal  is  not  included. 

Meanwhile,  Karch  and  Lasagna  (1975)  state  that  the  definition  could  be  interpreted  to 

include  this  classification,  but  recommend  it  not  be.  They  base  this  recommendation  on 

their  interpretation  that  the  failure  of  a  drug  to  produce  a  desired  effect  is  qualitatively 

different  than  the  production  of  an  undesirable  effect.  This  position  appears  somewhat 

irrational  as  the  failure  to  produce  the  desired  effect  is  by  definition  undesired. 

The  definition  of  an  adverse  drug  reaction  would  also  call  into  question  efforts  to 
prevent  occurrences  since  they  are  unexpected  and  occur  at  therapeutic  doses  (Lazarou  et 
al.,  1998;  Ross,  2001).  Preventability  did  not  receive  much,  if  any,  attention  in  the  early 
stages  of  defining  and  identifying  adverse  drug  reactions  (Karch  &  Lasagna,  1975). 

Preventability  of  adverse  drug  reactions  is  further  hindered  by  the  excessive  focus 
on  linking  a  specific  drug  entity  to  the  adverse  event.  This  is  best  illustrated  by  the 
algorithms  developed  to  assist  in  identifying  the  causative  agent  (Kramer  et  al.,  1979; 
Naranjo  et  al.,  1981).  Focusing  on  the  drug  as  the  primary  cause  (rather  than  the  use  of 
the  drug)  hinders  efforts  to  uncover  problems  in  the  medication  use  system.  Thus, 
without  a  systems  perspective,  improvement  activities  using  ADR  information  are 
limited. 

Some  researchers  have  addressed  the  preventability  of  ADRs.  Hallas  et  al.  (1990) 
looked  at  consecutive  admissions  to  assess  drug-related  hospitalizations.  They  used  the 
term  avoidability  rather  than  preventability.  Their  study  classified  events  as  definitely 
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avoidable,  possibly  avoidable,  not  avoidable,  or  unevaluable.  A  definitely  avoidable 

event  was  one  in  which 

the  drug  event  was  due  to  a  drug  treatment  procedure  inconsistent  with 
present-day  knowledge  of  good  medical  practice  or  was  clearly  unrealistic, 
taking  the  known  circumstances  into  account.  (Hallas  et  al.,  1990,  p.  85) 

Hallas  et  al.  (1990)  used  an  implicit  format  for  evaluating  the  causative  nature  of 
a  hospital  admission  and  its  avoidability.  Avoidability  was  assessed  only  in  events 
judged  to  have  been  definite  or  probable  in  their  causal  relationship  with  the  drug,  and 
where  the  symptoms  played  a  role  in  the  hospital  admission.  Their  use  of  present-day 
knowledge  in  evaluating  avoidability  gives  the  appearance  of  evidence-based  criteria. 
However,  the  use  of  physician  review  may  open  the  door  for  local  practice  standards  to 
override  evidence. 

Hallas  et  al.  (1990)  advocate  an  intensive  monitoring  program  by  suggesting  that 
a  valid  estimate  of  drug-related  hospitalizations  will  only  be  obtained  if  a  qualified 
individual  evaluates  each  case  independently.  However,  this  may  not  be  practical  or 
possible  in  most  organizations. 
Medication  Errors 

Medication  errors  are  receiving  more  attention  in  both  the  peer-review  and  lay 
press.  A  report  released  by  the  Institute  of  Medicine  (IOM)  in  1999  caused  politicians, 
health  care  professionals,  and  the  public  to  intensify  the  search  for  answers  to  this 
problem  (Institute  of  Medicine  Committee  on  Quality  Health  Care  in  America,  1999). 

Health  care  professionals  have  been  addressing  the  issue  of  medication  errors  for 
some  time.  In  fact,  entire  books  have  been  published  on  the  issue  (Cohen,  1999).  A 
medication  error  has  been  defined  as  "an  error  in  the  process  of  ordering  or  delivering  a 
medication"  (Cullen  et  al.,  1995,  p.  543).  In  ambulatory  patients,  this  could  be 
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interpreted  as  the  prescribing  or  dispensing  of  medications.  In  the  institutional  setting, 

this  would  also  include  the  administration  of  the  drug.  However,  neither  covers  the  entire 

medication  use  system  described  previously. 

Labeling  an  event  a  "medication  error"  may  lead  to  a  misdirected  search  for 
causes.  When  an  event  is  referred  to  as  an  error,  fixing  of  blame  (or  finding  fault)  on 
individuals  often  follows  (Berwick  &  Leape,  1999;  Reinersten,  2000).  Reinersten  (2000) 
supports  this  by  stating,  "deep  down  we  believe  that  individual  diligence  should  prevent 
error"  (p.  730).  Disciplinary  action  against  the  individual(s)  identified  in  the 
investigation  is  often  the  remedy  applied  to  fix  the  problem.  By  blaming  individuals  for 
the  adverse  event,  important  system  issues  necessary  for  long-term  improvement 
continue  unabated. 

Many  of  the  strategies  employed  to  correct  medication  errors  involve  viewing 
individual  process  components.  This  usually  takes  the  form  of  prescribing  improvement, 
new  dispensing  systems,  or  improving  patient  compliance  (Aparasu  &  Fliginger,  1997; 
Bergman  &  Wiholm,  1981;  Dartnell  et  al.,  1996;  Nightingale  et  al.,  2000).  Component 
management  operates  in  isolation  from  the  medication  use  system  as  a  whole  and  the 
intended  impact  may  never  be  realized. 
Adverse  Drug  Events 

The  term  adverse  drug  event  (ADE)  has  become  popular  due  to  a  series  of  articles 
by  researches  from  Harvard  University  (Bates  et  al.,  1995b;  Bates  et  al.,  1997;  Bates  et 
al.,  1995a;  Bates  et  al.,  1993;  Cullen  et  al.,  1995;  Leape  et  al.,  1991).  The  Harvard 
research  team  has  defined  an  adverse  drug  event  as  "an  injury  resulting  from  the  use  of  a 
drug"  (Cullen,  et  al.,  1995,  p.  541).  An  ADE  differs  from  an  ADR  in  that  it  relaxes  the 
"normal  dosage"  requirement  and  allows  for  causes  other  than  the  drug  entity.  The 
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definition  does  not,  however,  deal  with  the  lack  of  drug  use  when  medication  therapy  is 

clearly  indicated. 

The  work  done  by  the  Harvard  group  has  explored  the  idea  of  preventability.  In 
one  article,  Leape  et  al.  (1991)  suggests  that  adverse  effects  are  not  random  events  but  are 
system  dependent.  By  looking  at  problems  as  system  dependent,  causes  other  than  the 
drug  product  can  be  identified.  Despite  their  discussion  of  systems  issues,  much  of  what 
the  Harvard  group  terms  preventable  is  attached  to  a  finding  of  either  negligence  or  error 
(Cullen  et  al,  1995).  Negligence  was  defined  as  "care  that  fell  below  the  standard 
expected  of  physicians  in  their  community"  (Brennan  et  al.,  1991,  p.  370).  An  error  was 
defined  as  "a  mistake  in  performance  or  thought"  (Leape  et  al.,  1991,  p.  377).  By 
looking  for  either  negligence  or  error  when  assessing  preventability,  the  tendency  is  to 
once  again  place  blame  and  obscure  system  solutions. 

The  Harvard  group  employed  intensive  surveillance  techniques  using  a  research 
nurse  to  identify  adverse  drug  events  in  hospitalized  patients  (Bates  et  al.,  1995b;  Bates  et 
al.,  1993;  Leape  et  al.,  1991).  The  evaluation  of  identified  events  and  their  preventability 
was  accomplished  through  implicit  physician  review.  While  their  results  illuminate  the 
problem  of  ADEs,  this  method  would  be  nearly  impossible  to  replicate  in  other  settings. 
The  difficulty  in  replication  centers  on  two  issues  (Brennan,  2000).  First,  the  results  of 
an  implicit  evaluation  may  change  based  on  the  local  standards  of  practice  and  expert 
opinion.  Second,  this  method  would  be  cost  prohibitive  for  most  organizations. 
Drug-Related  Morbidity 

Drug-related  morbidity  (DRM)  is  a  term  I  recommend  for  use  to  describe  injury 
due  to  the  misuse  or  lack  of  use  of  medications.  Hepler  and  Strand  (1990)  have  defined 
drug-related  morbidity  as  the  "clinical  or  biosocial  manifestation  of  unresolved  drug- 
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related  problems"  (p.  535).  Thus,  drug-related  morbidity  is  preceded  by  a  drug-related 

problem. 

Drug-related  problems  are  events  or  circumstances  involving  drug  treatment  that 
actually  or  potentially  interfere  with  the  patient  achieving  an  optimal  outcome  of  medical 
care  (Hepler  and  Strand,  1990).  Eight  categories  of  drug-related  problems  have  been 
identified  by  Strand  et  al.  (1990).  They  include;  1)  untreated  indications,  2)  improper 
drug  selection,  3)  subtherapeutic  dose,  4)  failure  to  receive  drug,  5)  overdose,  6)  adverse 
drug  reaction,  7)  drug  interactions,  and  8)  drug  use  without  indication. 

Drug-related  problems  signal  a  pattern  of  care  that,  if  not  changed,  may  lead  to 
drug-related  morbidity.  These  drug-related  problems  may  serve  as  the  impetus  for 
monitoring  plans  to  determine  if  medications  are  being  used  appropriately.  Thus,  efforts 
directed  at  understanding  and  correcting  drug-related  problems  may  slow  or  eliminate 
drug-related  morbidity  through  changes  in  the  medication  use  system  at  the  earliest  sign. 
If  definitions  exist  that  identify  patterns  of  care  indicative  of  a  drug-related  problem 
(serving  as  an  indicator),  prevention  of  drug-related  morbidity  is  theoretically  possible. 
By  viewing  drug-related  problems  as  the  precursor  to  drug-related  morbidity,  the 
connection  of  preventability  to  a  finding  of  error  is  eliminated.  This  would  allow 
practitioners  to  move  beyond  blaming  simple  causes,  toward  a  quality  assessment  and 
improvement  philosophy  (Hepler  &  Grainger-Rousseau,  1995). 

Hepler  and  Strand  (1990)  have  proposed  a  method  to  evaluate  preventability  of 
drug-related  morbidity.  An  adverse  outcome  attributable  to  drug  therapy  can  be  deemed 
preventable  if 

(1)  the  adverse  outcome  was  preceded  by  a  recognizable  drug  therapy  problem, 
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(2)  the  adverse  outcome  of  the  drug  therapy  problem  had  been  reasonably 
foreseeable, 

(3)  the  cause  of  the  adverse  outcome  could  have  been  identifiable  with 
reasonable  probability,  and 

(4)  the  cause  of  the  adverse  outcome  could  have  been  reasonably  controllable 
within  the  context  and  objectives  of  therapy. 

Hepler  and  Strand's  (1990)  criteria  for  preventability  offer  a  framework  to 
develop  explicit  criteria  that  measure  preventable  drug-related  morbidity.  This  would  be 
offered  in  contrast  to  the  labor-intensive  and  expensive  use  of  experts  currently  relied 
upon  in  the  literature. 
Study  Populations 

Many  of  the  studies  published  in  the  literature  focus  on  events  as  they  relate  to 
hospitals  or  other  institutional  care.  The  focus  on  hospitals  takes  a  limited  view  of  the 
total  amount  of  harm  being  done  to  patients  (Brennan,  2000).  The  problems  associated 
with  outpatient  care  could  be  far  more  significant,  although  also  more  difficult  to  study 
and  quantify  ( Weingart  et  al.,  2000).  There  has  not  been  a  concerted  effort  to  assess 
actual  adverse  drug  reactions  (and  thus,  drug-related  morbidity)  in  ambulatory 
populations  (Karch  &  Lasagna,  1975). 

Research  on  adverse  events  in  ambulatory  patients  is  just  beginning  to  garner 
attention.  The  volume  of  patients  receiving  care  outside  the  hospital  justifies  the 
attention.  Between  1983  and  1993,  outpatient  visits  increased  75%  while  inpatient  days 
declined  21%  (Phillips  et  al.,  1998).  One  study  suggests  that  ambulatory  patients  have  an 
adverse  drug  reaction  rate  of  2.5%,  compared  with  an  inpatient  rate  of  9%  (Burnum, 
1976).  While  the  percentage  for  ambulatory  patients  may  be  lower,  the  actual  number 
experiencing  an  adverse  event  is  likely  higher  due  to  the  volume  of  patient  encounters. 


24 
It  is  also  likely  that  ambulatory  patients  die  from  drug  therapy  at  a  rate  not  yet 

fully  understood.  Early  data  suggest  that  outpatient  deaths  would  greatly  increase  current 

estimates  (Leape,  2000).  These  factors  support  additional  efforts  directed  at  measuring 

preventable  drug-related  morbidity  in  ambulatory  patients. 

Measurement 

Inconsistency  of  surveillance  methodologies  hinders  the  comparison  of  results 
across  studies  (Ross,  2001).  Previously  mentioned  studies  show  large  variations  in  their 
results.  While  some  variation  is  due  to  different  patient  populations,  much  may  be  due  to 
different  surveillance  techniques  or  definitions  (Kramer  et  al.,  1979;  Ross  2001).  There 
may  also  be  differences  in  the  scope  of  events  monitored  or  the  intensity  of  data 
collection  (Caranasos  et  al.,  1974;  Hallas  et  al.,  1990).  There  is  a  need  for  consistency  in 
both  the  terminology  and  methodology  of  studies  investigating  preventable  drug-related 
morbidity  (Weingart  et  al.,  2000;  Ross  2001). 

The  screening  mechanisms  employed  in  the  literature  can  be  classified  into  one  of 
three  categories:  implicit,  structured  implicit,  and  explicit  (Ashton  et  al.,  1999).  Each  is 
described  and  examples  from  the  literature  given. 
Implicit 

An  implicit  review  in  essence  is  peer  review,  with  experts  comparing  the  level  of 
care  provided  to  their  own  knowledge,  opinions,  and  beliefs.  The  literature  has  many 
examples  of  studies  where  implicit  judgement  was  used  to  determine  the  presence  of 
drug-related  morbidity  (Bates  et  al.,  1995b;  Bates  et  al.,  1993;  Bero  et  al.,  1991;  Bigby  et 
al.,  1987;  Hallas  et  al.,  1992). 

Adverse  drug  reaction  studies  have  used  implicit  judgement  for  years.  Kramer  et 
al.  (1979)  state  that 
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the  diagnosis  of  an  ADR  has  usually  depended  on  unspecific  clinical 
judgement,  arising  from  the  subjective  impression  and  previous 
experience  of  individual  clinicians,  (p.  623) 

Thus,  implicit  review  tends  to  be  highly  reviewer  dependent  and  often  has  difficulty 
establishing  significant  conclusions  due  to  the  broad  responses  received  from  reviewers 
(Ashton  et  al.,  1999;  Karch  et  al.,  1976;  Thomas  &  Brennan,  2000). 

There  are  also  suggestions  that  implicit  review  is  so  subjective  as  to  be  invalid 
(Donabedian,  1978;  Lipton  &  Bird,  1993).  This  may  be  especially  true  when  an  expert 
views  the  entire  chart  with  the  outcome  known  in  advance  (Caplan  et  al.,  1991).  Brennan 
(2000)  states  that  physician  judgements  in  the  Harvard  study  may  not  be  reliable.  Other 
authors  have  also  taken  this  position  (Thomas  &  Brennan,  2000).  One  attempt  to 
replicate  the  Harvard  study  resulted  in  drastically  different  results  (Wilson  et  al.,  1995). 
Additionally,  implicit  screening  is  very  expensive  and  time  consuming,  limiting  its 
usefulness  for  most  health  care  organizations  (Donabedian,  1 978). 
Structured  Implicit 

A  second  measurement  technique  is  referred  to  as  a  structured  implicit  review.  A 
structured  implicit  review  combines  the  expert's  internalized  standards  with  directions 
that  lead  them  to  look  at  specific  issues  in  care  on  which  judgements  are  to  be  made 
(Ashton  et  al.,  1999). 

A  common  use  for  structured  implicit  review  has  been  in  the  area  of  adverse  drug 
reactions.  Both  the  Naranjo  et  al.  (1981)  and  Kramer  et  al.  (1979)  algorithms  use  a 
structured  process  to  assist  experts  in  judging  the  causality  of  adverse  drug  reactions. 
Each  algorithm  provides  specific  questions  that  direct  practitioners  to  gather  evidence  in 
determining  the  probability  of  an  ADR. 
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A  structured  implicit  review  may  increase  the  reliability  of  the  measure.  This  is 

due  to  the  directing  of  experts  to  look  for  certain  information  in  the  patient's  medical 

record.  However,  this  method  is  still  labor-intensive  and  costly. 

Explicit 

The  final  method  of  screening  for  drug-related  events  uses  explicit  criteria. 

Explicit  criteria  compare  the  actual  care  process  against  a  set  of  statements  or  criteria  that 

have  been  previously  developed  (Ashton  et  al.,  1999).  This  method  of  surveillance  is 

highly  reliable  and  has  good  predictive  validity  (Ashton  et  al.,  1999;  Caplan  et  al.,  1991). 

These  are  good  characteristics  for  indicators  of  quality  problems  in  the  medication  use 

system. 

In  addition  to  good  measurement  properties,  practical  support  exists  for  the 
further  development  and  testing  of  explicit  criteria.  Explicit  criteria  can  be  assembled 
into  quality  measures  independent  of  the  chart  review  process  (Geraci  et  al.,  1999).  This 
allows  the  criteria  to  serve  as  an  ongoing  and  consistent  quality  measure  that  does  not 
rely  on  chart  review  and  abstraction.  Recent  recommendations  by  the  Institute  of 
Medicine  (1999)  have  listed  the  development  of  explicit  criteria  for  measuring  adverse 
medication  events  as  a  priority. 

Despite  the  potential  value  of  explicit  criteria,  there  are  some  pitfalls.  Explicit 
criteria  are  only  as  good  as  the  development  process.  This  can  place  criteria  at  one  of  two 
extremes;  they  can  either  be  from  leading  experts  using  evidence-based  literature,  or  they 
can  reflect  the  average  practice  of  local  physicians  (Donabedian,  1978).  The  burden  of 
accuracy  falls  on  the  criteria  and  the  biggest  challenge  is  assuring  that  standards  to  which 
criteria  are  compared  are  valid  (Ashton  et  al.,  1999).  Explicit  criteria  do  not  take  into 
account  individual  patient  care  variability  (Donabedian,  1978).  This  could  be  a 
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stumbling  block  for  physician  acceptance  of  explicit  criteria  to  evaluate  the  quality  of 

medication  use  if  the  organization  uses  them  in  a  punitive  manner. 

Explicit  criteria  to  evaluate  prescribing  in  nursing  home  patients  have  been 
developed  by  Beers  et  al.  (1991).  These  criteria  were  recently  updated  to  include  a 
broader  classification  of  elderly  patients  (Beers,  1997).  The  work  provides  the 
opportunity  to  evaluate  the  prescribing  of  medications  using  explicit  criteria  developed 
by  an  expert  panel.  However,  they  only  evaluated  whether  the  drug  was  appropriate  or 
not  based  on  a  global  indicator.  No  attempt  was  made  to  incorporate  clinical  information 
or  outcomes  data. 

Hanlon  et  al.  (1992)  attempted  to  develop  explicit  criteria  as  an  index  of 
appropriate  medication  use  in  the  elderly.  They  used  a  ten-item  scale  that  judged  the 
decision  making  on  a  broader  scale  than  Beers  et  al.  (1991).  However,  there  was  no  link 
to  the  patient  outcome  to  determine  the  usefulness  of  the  measure. 

A  recent  study  by  MacKinnon  (1999)  developed  definitions  for  preventable  drug- 
related  morbidity  (PDRM)  that  linked  the  process  and  outcome.  These  definitions  may 
be  operationalized  into  explicit  criteria  and  used  to  measure  preventable  drug-related 
morbidity  with  available  information  system  technology.  The  strength  of  MacKinnon's 
(1999)  work  is  the  ability  to  detect  both  drug-related  problems  and  adverse  outcomes 
together.  A  measure  of  the  outcome  has  been  suggested  as  the  only  way  to  know  is 
medication  use  is  appropriate  (Buetow  et  al.,  1996). 
Use  of  Computer  Databases 

The  use  of  computers  to  store  health  care  information  has  shifted  from  a  financial 
focus  to  incorporate  more  clinical  data.  This  shift  offers  additional  research  opportunities 
using  administrative  databases.  One  author  has  suggested  more  funding  to  research 
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secondary  databases  and  less  using  primary  data  sources  (Kralewski  et  al.,  1994).  In 

addition,  the  Institute  of  Medicine  (1999)  report  has  recommended  the  use  of  automation 

whenever  possible.  These  factors  support  extensive  growth  in  research  using  patient  care 

databases. 

Administrative  data  are  being  used  to  evaluate  care  (Paul  et  al.,  1993;  Romano  & 
Mark,  1994).  Administrative  databases  have  advantages  over  primary  data  collection  by 
allowing  a  more  efficient  review  of  a  large  number  of  cases,  longitudinal  data  collection, 
long-term  follow-up,  a  reduction  in  the  cost  of  data  collection,  defined  sampling  frames, 
and  the  linkage  of  several  databases  (Jollis  et  al.,  1993;  Roos  et  al.,  1991;  Steinberg  et  al., 
1990). 

Administrative  databases  also  have  their  drawbacks.  The  databases  often  do  not 
include  all  the  details  involved  in  caring  for  patients,  which  may  cause  information  to  be 
lost  (Roos  et  al.,  1991).  In  addition,  the  accuracy  of  the  database  may  suffer  if 
information  is  either  entered  incorrectly  or  not  entered  at  all  (Romano  &  Luft,  1992). 
Despite  these  limitations,  administrative  databases  have  an  important  role  in  studying  the 
outcomes  of  care  (Romano  &  Luft,  1992). 

The  use  of  computers  to  screen  patients  for  adverse  events  is  receiving  more 
attention  as  computer  capabilities  increase  (Raschke  et  al.,  1998).  Computer  screening  of 
patient  populations  has  been  shown  to  detect  more  events  than  spontaneous  reporting 
mechanisms  currently  in  place  (Bates  et  al.,  1995;  Evans  et  al.,  1994).  Automated 
screenings  have  also  been  shown  to  be  more  consistent  than  expert  ratings  (Camacho  & 
Rubin,  1 998).  This  could  lead  to  consistency  between  organizations  and  allow 
benchmarking  for  future  improvement  initiatives.  Thus,  well-developed  explicit  criteria 
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used  with  an  automated  database  may  hold  the  key  to  system  monitoring  and  ultimately 

for  medication  use  improvements. 

The  use  of  computers  to  screen  for  preventable  drug-related  morbidity  will  also 
prove  cost-effective.  Rosenheck  et  al.  (1999)  estimated  that  in  a  Veterans  Administration 
(VA)  patient  population  it  would  cost  between  $150-300  per-patient-per-year  to 
intensively  monitor  for  adverse  outcomes.  This  would  add  $174  million  per  year  to  the 
VA  budget  for  screening  alone.  If,  however,  there  were  valid,  explicit  criteria  that  could 
be  used  with  current  computer  technology,  this  cost  would  drop  to  $0.25  per-patient-per- 
year  (Rosenheck  et  al.,  1999).  This  cost  difference  would  be  significant,  supporting  the 
use  of  computers  and  explicit  criteria  in  patient  care  research. 

Early  evidence  suggests  that  computers  can  make  a  difference  when  monitoring 
patient  events  (Bates  et  al.,  1995b;  Bates  et  al.,  1993;  Christensen  &  Penna,  1995; 
Classen  et  al.,  1991).  These  studies  demonstrate  that  the  use  of  computers  with  explicit 
definitions  to  measure  preventable  drug-related  morbidity  is  possible.  However,  the  next 
question  will  be  assessing  the  validity  and  use  of  the  measures. 

Validity 

Validity  is  an  integrated  evaluative  judgement  of  the  degree  to  which  empirical 
evidence  and  theoretical  rationales  support  the  adequacy  and  appropriateness  of 
inferences  and  actions  based  on  measurement  or  other  modes  of  assessment  (Messick, 
1989).  In  terms  of  drug-related  morbidity  as  an  indicator  of  the  quality  of  medication 
use,  validity  is  the  degree  to  which  the  data  indicating  preventable  drug-related  morbidity 
identifies  situations  in  which  the  quality  of  medication  use  is  poor  and  can  be  improved 
(Nadzam,  1991 ).  Rigorous  assessment  of  validity  in  evaluating  drug-related  events  has 


30 

not  received  much  attention  in  the  literature  (Ross,  2001).  Several  issues  related  to 

validity  are  relevant  to  this  discussion  and  are  addressed  in  the  pages  that  follow. 
Triarchic  Validity 

The  concept  of  validity  has  been  evolving  since  the  1950s.  Early  in  its 
development,  validity  was  viewed  as  having  separate  components.  These  can  be 
classified  as  content  validity,  criterion-related  validity,  and  construct  validity. 

Content  validity  is  based  on  professional  judgements  about  the  relevance  of  the 
content  to  a  particular  domain  of  interest,  and  about  how  completely  and  precisely  the 
items  or  tasks  represent  that  domain.  Content  validity  is  evaluated  by  showing  how  well 
the  content  samples  the  class  of  situations  about  which  conclusions  are  to  be  drawn 
(Messick,  1 989).  Content  validity  does  not  measure  actual  performance,  but  rather  is 
only  a  judgement  based  on  expert  opinion. 

The  second  type  of  validity  is  criterion-related  validity.  Criterion-related  validity 
is  evaluated  by  comparing  the  measurement  results  with  one  or  more  external  variables 
(called  criterion)  that  provide  a  direct  measure  of  the  characteristic  in  question  (Messick, 
1989).  The  criterion  is  an  existing  "gold  standard"  by  which  the  proposed  measure  is 
compared.  Criterion-related  validity  is  based  on  the  degree  of  empirical  relationship 
(correlation  or  regression)  between  the  test  scores  and  criterion  scores.  There  are  as 
many  criterion-related  validity  measures  as  there  are  criterion. 

Criterion-related  validity  can  be  divided  into  predictive  and  concurrent  validity. 
Predictive  validity  indicates  the  extent  to  which  future  performance  correlates  with  the 
current  measure.  Concurrent  validity  indicates  the  degree  to  which  the  measure  estimates 
the  present  standing  on  the  criterion. 
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The  third  validity  category  is  construct  validity.  Construct  validity  is  evaluated 

by  investigating  the  qualities  a  test  measures.  Construct  validity  attempts  to  determine 

the  degree  to  which  certain  explanatory  concepts  account  for  the  measured  results 

(Messick,  1989).  Construct  validity  may  use  any  evidence  that  affects  the  interpretation 

or  meaning  of  the  test  scores.  Almost  any  kind  of  information  about  a  test  may 

contribute  to  an  understanding  of  construct  validity,  but  is  stronger  if  the  degree  of  fit  of 

the  information  with  the  theoretical  rationale  underlying  score  interpretation  is  explicitly 

evaluated. 

Messick' s  Validity  Framework 

Validity  has  evolved  into  a  unitary  concept  (Messick,  1989).  This  evolution  does 
not  allow  validity  to  be  neatly  packaged  into  three  different  categories  that  serve  as 
independent  indicators  of  validity.  The  prevailing  thought  of  having  separate  validity 
components  has  lead  many  (especially  in  health  care)  to  use  expert  judgment  of  content 
as  their  sole  assessment  of  validity,  and  thus  to  conclude  that  the  instrument  is  valid  for 
any  use  from  that  point  forward. 

Validity  must  involve  a  measure  and  an  intended  use.  Without  a  measure,  there  is 
no  assessment  of  validity  (Messick,  1989).  When  validating  a  measure,  what  is  being 
tested  is  not  the  observation  device,  but  the  inferences  about  the  meaning  of  the  score  and 
the  implications  for  actions  taken  based  on  that  score. 

Validity  is  also  not  complete  after  one,  or  even  a  few  observations.  Rather, 
validity  is  a  matter  of  degree  and  builds  over  time  as  evidence  gathers  either  for  or 
against  the  particular  use  or  observation.  Thus,  validation  is  essentially  a  matter  of 
making  the  most  reasonable  case  to  guide  both  the  current  use  of  a  measure  and  research 
to  advance  understanding  of  what  the  score  means. 


:>: 


According  to  Messick  (1989), 


what  is  needed  is  a  way  of  cutting  and  combining  validity  evidence  that 
forestalls  undue  reliance  on  selected  forms  of  evidence,  that  highlights  the 
important  though  subsidiary  role  of  specific  content-  and  criterion-related 
evidence  in  support  of  construct  validity  in  testing  applications,  and  that 
formally  brings  consideration  of  value  implications  and  social 
consequences  into  the  validity  framework,  (p.  20) 

Messick  (1989)  attempts  to  meet  this  challenge  by  proposing  a  unified  validity 
framework  distinguishing  two  interconnected  facets.  One  facet  is  the  source  of 
justification  of  the  measure  (primarily  based  on  appraisal  of  either  evidence  or 
consequence).  The  second  facet  is  the  function  or  outcome  of  the  measure,  being  either 
interpretation  or  use.  Messick  (1989)  proposed  a  two-by-two  table  in  which  each  block  is 
linked  to  the  information  used  to  satisfy  it  (see  Figure  2.3). 

The  evidential  basis  for  interpretation  of  a  measure  requires  what  we  commonly 
refer  to  as  construct  validity.  The  evidential  basis  of  the  use  of  a  measure  also  looks  to 
construct  validity,  but  is  supported  by  evidence  for  the  relevance  of  the  measure  to  the 
specific  purpose  (i.e.,  content  validity)  and  for  the  utility  of  the  measure  in  the  applied 
setting.  Because  content  and  criterion-related  validity  contribute  to  score  meaning,  they 
have  come  to  be  recognized  as  aspects  of  construct  validity  (Messick,  1989). 


Interpretation 


Use 


Evidential 
Basis 


Consequential 
Basis 


Construct  validity 

Construct  validity  + 
relevance/utility 

Value  implications 

Social  consequences 

Figure  2.3:  Messick's  Facets  of  Validity  (Messick,  1989,  p. 20) 
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The  consequential  basis  for  interpretation  of  a  measure  is  an  appraisal  of  the  value 

implications  of  the  construct  label,  of  the  theory  underlying  the  interpretation,  and  the 

ideologies  in  which  the  theory  is  imbedded.  A  central  issue  is  whether  or  not  the 

theoretical  implications  and  the  value  implications  of  the  interpretation  of  a  measure  are 

commensurate,  realizing  that  value  implications  are  tied  to  score  meaning. 

The  consequential  basis  for  the  use  of  the  measure  is  the  appraisal  of  both 
potential  and  actual  social  consequences.  The  social  consequences  are  important  to 
anything  we  are  validating  that  may  have  an  impact  on  society  or  policy.  Value  and 
social  consequences  are  rarely  considered  when  assessing  validity. 

Preventable  Drug-Related  Morbidity  and  Validity 

When  assessing  the  validity  of  a  measure  of  preventable  drug-related  morbidity, 
evidence  can  be  gained  through  theoretical  links  that  have  been  established  in  the 
literature.  Risk  factors  for  patients  who  experience  preventable  drug-related  morbidity 
have  been  studied.  Risk  factors  are  variables  that  are  statistically  related  to  the  adverse 
event.  Risk  factors  that  are  related  to  the  presence  of  PDRM  should  be  more  prevalent  in 
patients  with  an  instance  of  preventable  drug-related  morbidity  than  in  those  patients 
without  an  instance.  The  following  items  represent  risk  factors  identified  in  the  literature 
and  shown  to  be  significant  in  MacKinnon's  (1999)  study. 
Number  of  Medications 

The  number  of  medications  a  patient  receives  has  been  studied  on  several 
occasions.  Studies  suggest  that  the  more  drugs  a  patient  is  taking,  the  more  likely  they 
are  to  experience  an  adverse  event  (Classen  et  al.,  1991;  Colley  &  Lucas,  1993;  Gandhi  et 
al.,  2000;  Hallas  et  al.,  1992;  Hanlon  et  al.,  1997;  Hurwitz,  1969;  Lakshmanan  et  al., 
1986;  MacKinnon,  1999;  Smith  et  al.,  1966). 
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Number  of  Prescribers 

The  number  of  prescribers  has  been  show  to  be  a  risk  factor  by  Colley  and  Lucas 
( 1 993)  and  MacKinnon  ( 1 999).    A  patient  with  more  prescribers  has  a  higher  risk  for 
PDRM.  This  may  be  due  to  poor  communication  between  prescribers  and  a  lack  of 
information  on  the  part  of  each  prescriber. 
Cardiovascular  Drugs 

Studies  by  Hanlon  et  al.  (1997),  Lindley,  et  al.  (1992)  and  Bates  et  al.  (1999) 
identify  cardiovascular  drugs  as  a  risk  factor  for  adverse  events.  Escobedo  and  Zach 
(1996)  and  MacKinnon  ( 1 999)  specify  the  category  of  antihypertensive  drugs  as 
significantly  associated  with  adverse  events. 
Gender 

The  issue  of  gender  is  less  clear.  Studies  by  Hallas  et  al.  (1992),  Lakshmanan  et 
al.  (1986)  and  Hurwitz  (1969)  state  that  females  are  more  likely  to  experience  an  adverse 
event.  This  is  in  direct  contrast  to  the  MacKinnon  (1999)  study,  which  identified  males 
as  being  more  likely  to  experience  preventable  drug-related  morbidity. 
Number  of  Diagnoses 

The  number  of  diagnoses  has  also  been  identified  as  a  risk  factor.  Studies 
indicate  that  a  patient  with  more  co-morbidities  has  a  higher  likelihood  of  experiencing 
an  adverse  event  (Gandhi  et  al.,  2000;  MacKinnon,  1999;  Smith  et  al.,  1966;  Trunet  et  al, 
1980). 
Age 

Age  has  been  identified  in  previous  studies  as  a  risk  factor  for  adverse  drug  events 
(Brennan  et  al.,  1991;  Classen  et  al.,  1991;  Colley  &  Lucas,  1993;  Hallas  et  al.,  1992; 
Hanlon  et  al.,  1997;  Hurwitz,  1969;  Lakshmanan  et  al.,  1986;  Thomas  &  Brennan,  2000; 
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Weingart  et  al.,  2000).  When  studying  an  elderly  population,  chronological  age  may  not 

be  as  important.  However,  patients  over  85  (oldest  of  the  old)  may  be  of  interest.  It  has 
been  shown  that  patients  over  85  are  at  a  higher  risk  of  PDRM  than  those  between  65-85 
(Gurwitz  &  Avorn,  1991). 
Health  Care  Resource  Utilization  and  Cost 

Studies  indicate  that  drug-related  morbidity  increases  the  utilization  of  health  care 
resources  and  the  cost  of  caring  for  patients  (Bates  et  al.,  1993;  Dartnell  et  al.,  1996; 
Hanlon  et  al.,  1997;  Karch  &  Lasagna,  1975;  Lakshmanan  et  al.,  1986;  Ornstein  et  al., 
1999;  Tafreshi  et  al.,  1999;  Thomas  et  al.,  1999).  Thus,  patients  with  preventable  drug- 
related  morbidity  should  access  the  health  care  system  more  (in  terms  of  hospitalizations, 
emergency  room  visits,  physician  office  visits  and  medication  use)  than  patients  without 
PDRM.  In  addition,  patients  with  PDRM  should  have  higher  direct  medical  costs  than 
those  without.  This  cost  could  be  incurred  by  the  health  care  system  (thus,  the  payer)  or 
from  the  patient  in  the  form  of  co-pays  and  deductibles. 


CHAPTER  3 
METHODOLOGY 

Introduction 

The  study  was  divided  into  four  phases  to  accomplish  the  objectives  listed  in 
Chapter  1.  The  first  phase  of  the  study  involved  defining  specific  scenarios  of 
preventable  drug-related  morbidity  (PDRM).  The  second  phase  operationalized  the 
specific  definitions  of  PDRM  for  application  to  a  computer  database.  Phase  III  measured 
preventable  drug-related  morbidity  in  an  elderly  population  and  gathered  evidence  for 
validity  of  the  definitions.  The  final  phase  used  the  measure  of  PDRM  in  a  root  cause 
analysis  to  determine  their  usefulness  in  quality  assessment  and  improvement. 

All  four  phases  of  our  study  were  accomplished  with  the  aid  of  a  large  managed 
care  organization  (MCO).  The  managed  care  organization  was  selected  because  of  their 
elderly  patient  population  (a  Medicare  product),  available  database,  and  patient 
information  across  the  continuum  of  care.  Additionally,  the  managed  care  organization 
was  interested  in  new  methods  of  evaluating  drug  use  to  improve  care,  enhance  their 
position  in  the  market,  and  bolster  their  accreditation  survey  preparation. 
Phase  I:  Defining  Preventable  Drug-Related  Morbidity 
Delphi  Panel 

Phase  I  of  the  study  employed  a  Geriatric  Medicine  Expert  Panel  to  define 
explicit  scenarios  of  preventable  drug-related  morbidity.  The  expert  panel  used  the 
Delphi  technique  to  come  to  consensus  on  proposed  definitions  of  PDRM.  The  Delphi 
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technique  was  developed  in  the  1948  by  the  Rand  Corporation  as  a  means  to  obtain 

expert  opinion  in  a  systematic  manner  (Fink  et  al.,  1984). 

The  Delphi  technique  eliminates  interpersonal  interactions  as  the  controlling 
variable  in  group  decision  making  (Goodman,  1987).  The  intent  is  to  foster  discussion 
and  judgement  on  preventable  drug-related  morbidity  so  the  decisions  made  represent  the 
panel's  views.  By  using  the  Delphi  technique  with  a  panel  of  physicians,  the  drawbacks 
of  traditional  committee  structures  are  counteracted,  including  a  domineering  personality, 
the  unwillingness  to  be  seen  as  disagreeing,  and  the  vocal  minority  (Goodman,  1987). 

The  selection  of  panel  members  is  critical  if  the  Delphi  panel  is  to  function 
properly  (Duffield,  1993;  Fink  et  al.,  1984).  A  total  of  seven  (7)  physicians  were  chosen 
for  the  panel  (see  Appendix  A).  The  managed  care  organization's  regional  medical 
director  and  principal  investigator  (RJF)  selected  panel  members.  Physicians  were 
chosen  based  on  their  expertise  in  geriatrics,  practice  with  the  managed  care 
organization's  Medicare  patients,  and  their  willingness  to  participate. 

During  the  survey  process,  it  became  necessary  to  disqualify  one  panel  member 
(see  Chapter  4).  The  disqualification  was  based  on  the  physician's  lack  of  support  for  the 
project.  This  determination  was  made  after  reviewing  the  physician's  response  to  the 
surveys  and  a  face-to-face  meeting  with  the  principal  investigator  (RJF).  Another 
physician  was  selected  as  a  replacement. 
MacKinnon's  Definitions 

The  specific  scenarios  for  preventable  drug-related  morbidity  built  upon  previous 
work  done  by  MacKinnon  (1999).  MacKinnon  (1999)  used  literature  review  and  an 
expert  panel  to  create  52  operational  definitions  of  preventable  drug-related  morbidity. 
All  definitions  have  two  components:  a  pattern  (i.e.,  process)  of  care  and  an  associated 
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adverse  outcome.  MacKinnon's  (1999)  fifty-two  definitions  served  as  the  content  for  the 

first  survey.  In  addition  to  the  52  definitions,  three  scenarios  rejected  by  MacKinnon's 

(1999)  panel  were  included  in  the  original  Delphi  survey.  The  original  content  for  all 

definitions  from  MacKinnon's  (1999)  panel  remained  unchanged  for  the  first  survey. 

Panel  members  were  asked  to  vote  (yes  or  no)  whether  they  believed  a  certain 
scenario  represented  an  instance  of  preventable  drug-related  morbidity  (see  Appendix  B 
for  complete  instructions  given  to  panel  members).  The  results  of  each  survey  were 
compiled  and  comments  made  in  support  of  their  decision  were  recorded.  The  voting 
results  and  comments  were  used  to  in  subsequent  rounds.  Scenarios  receiving  the 
majority  of  panel  member's  support  were  included  in  the  next  survey.  The  process 
continued  until  consensus  did  not  change  appreciably. 

Phase  11:  Operalionalizinu  Definitions 

Phase  II  of  the  study  focused  on  operationalizing  the  consensus-approved 
definitions  for  measurement  in  an  administrative  database. 

The  managed  care  organization  (MCO)  uses  computer  technology  to  gather 
patient  encounter  information  from  physician  offices,  hospitals,  pharmacies,  and  other 
care  sites  in  their  network.  Information  is  communicated  between  the  care  site  and  the 
MCO  using  a  standardized  coding  system.  The  coding  of  health  care  encounters  is 
accomplished  through  a  set  of  standards  called  the  International  Classification  of 
Diseases-91  edition  (ICD-9)  and  Current  Procedural  Terminology  (CPT)  codes  (Banks  et 
al.,  1999;  Harris  &  Gordy,  1998).  ICD-9  coding  uses  the  diagnosis  as  its  base  for 
recording  encounters.  So-called  diagnostic  coding  is  the  primary  means  in  today's  health 
care  environment  for  submission  of  patient  care  claims.  ICD-9  codes  are  available  from 
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hospitals,  emergency  rooms,  physician  offices,  long-term  care  facilities,  and  skilled 

nursing  facilities. 

Current  Procedural  Terminology  (CPT)  codes  are  based  on  ambulatory  services 
rendered  to  a  patient,  often  in  a  physician's  office.  CPT  coding  is  a  systematic  listing  of 
procedures  and  services  performed  by  physicians  (Harris  &  Gordy,  1998).  CPT  codes 
are  specific  to  the  actual  services  a  patient  receives  at  an  ambulatory  care  site.  These 
codes  are  also  communicated  between  the  care  sites  and  the  MCO. 
Medical  Event  Coding 

The  current  system  used  to  code  health  care  encounters  is  very  complex 
(Christensen  &  Penna,  1995).  This  coding  system  was  designed  years  ago  with  financial 
reimbursement  as  the  primary  function  (Diehr  et  al.,  1999).  The  focus  on  financial 
aspects  has  hindered  the  utility  of  codes  for  other  activities,  such  as  clinical  evaluation  or 
research.  When  using  coded  data  for  reasons  other  than  financial,  outside  expertise  is 
needed  to  understand  and  overcome  inherent  difficulties.  The  recommended  procedure  is 
to  use  experienced  medical  record  coders  and  a  physician  or  other  clinical  expert 
(Steinberg  et  al.,  1990). 

Operationalizing  the  definitions  properly  is  crucial  for  application  to  the  managed 
care  organization's  database.  It  is  important  that  the  codes  selected  for  each  definition  be 
accurate  and  complete.  Two  medical  record  coders  were  chosen  to  operational ize  the 
definitions  for  the  study.  The  medical  record  coding  personnel  (coders)  were  chosen 
based  on  their  training,  expertise,  and  willingness  to  participate.  A  medical  record 
administrator  employed  by  a  university  hospital  assisted  in  the  selection  of  the 
participants. 
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Each  coder  was  given  the  49  definitions  agreed  upon  by  the  Geriatric  Medicine 

Expert  panel.  The  coders  worked  independently  to  select  all  appropriate  codes  (ICD-9 

and  CPT).  The  coders  were  instructed  to  assign  the  ICD-9  code  with  the  least  number  of 

digits  possible  to  properly  identify  the  pattern  or  outcome  of  care.  This  is  done  to  reduce 

coding  biases  (Romano  &  Luft,  1992;  Steinberg  et  al.,  1990). 

The  principal  investigator  (RJF)  compiled  a  list  of  all  codes  chosen  by  at  least  one 
coder.  For  each  definition,  the  percent  agreement  between  the  coders  was  calculated. 
The  percent  agreement  equals  the  number  of  codes  identified  by  both  coders  divided  by 
the  total  number  of  codes.  An  overall  mean  and  standard  deviation  for  the  coding  process 
was  calculated  based  on  the  percent  agreement  for  each  definition. 

All  codes  selected  by  the  coders  were  sent  to  a  physician  for  review.  The 
physician  was  instructed  to  review  each  definition  and  all  codes.  He  assigned  a  value  to 
each  code  based  on  its  consistency  with  the  definition.  The  following  three-point  scale 
was  used;  1)  the  code  is  definitely  consistent  with  the  definition,  2)  the  code  is  possibly 
consistent  with  the  definition,  or  3)  the  code  is  not  consistent  with  the  definition.  Codes 
receiving  a  1  or  2  were  collapsed  into  a  category  "consistent  with  the  definition."  Codes 
that  received  a  score  of  3  were  eliminated.  The  final  ICD-9  and  CPT  codes  remaining 
were  used  to  measure  PDRM  in  the  patient  population. 
Medication  Coding 

Medications  were  coded  for  each  definition  using  National  Drug  Codes  (NDC). 
Lexicon"  from  Multum  (www.multum.com)  was  used  to  obtain  all  NDC  numbers  for  the 
targeted  medications.  Multum  provided  a  mechanism  to  identify  all  appropriate  NDC 
numbers  based  on  the  individual  medication  or  class  of  medications. 
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One  of  the  risk  factors  in  the  study  was  "patients  receiving  a  cardiovascular 

medication."  Because  of  the  broad  nature  of  this  category  and  the  number  of  NDC 

numbers  that  would  be  required,  a  method  other  than  NDC  codes  was  used.  The 

managed  care  organization's  database  attaches  an  American  Hospital  Formulary  Service 

(AHFS)  number  to  each  medication.  The  AHFS  classification  number  is  a  systematic 

coding  mechanism  for  classes  of  medications.  One  such  class  is  cardiovascular  drugs, 

which  allowed  easy  identification  of  the  targeted  medications.  AHFS  numbers  were  only 

used  for  this  risk  factor. 

Phase  III:  Measurement  of  Preventable  Drug-Related  Morbidity  and  Validity 

Phase  III  of  the  study  measured  the  prevalence  of  preventable  drug-related 
morbidity  in  an  ambulatory  Medicare  population.  Additionally,  evidence  to  validate  the 
definitions  was  gathered. 
Studv  Period  and  Population 

The  data  collection  period  totaled  18  months,  from  April  1,  1998  to  September  30, 
1999.  The  study  population  consisted  of  all  patients  over  the  age  of  65  who  were 
enrolled  in  the  managed  care  organization's  Medicare  product  for  at  least  12  months  of 
the  study  period.  The  elderly  were  chosen  for  study  due  to  their  higher  drug  use  and  the 
higher  likelihood  of  drug-related  problems  (Caranasos  et  al.,  1974;  Culler  et  al.,  1998; 
Hurwitz,  1969). 

The  elderly  may  be  particularly  well  suited  to  using  administrative  data.  Claims 
records  capture  clinical  conditions  more  often  in  patients  over  64  than  in  those  64  years 
of  age  or  less  (Jollis  et  al.,  1993).  This  is  primarily  due  to  the  presence  of  chronic 
conditions.  Data  quality  has  also  been  shown  to  improve  as  more  data  points  are 
collected  over  a  longer  period  of  time  (Roos  et  al.,  1991). 
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Managed  Care  Database 

The  managed  care  organization  gathers  data  across  the  entire  continuum  of  care 
(hospitalizations,  physician  office  visits,  emergency  room  encounters,  drugs,  etc.). 
Gathering  information  across  the  continuum  of  care  strengthens  the  study.  Many  elderly 
patients  have  chronic  conditions  that  may  not  require  hospitalization  for  a  long  period  of 
time.  Thus,  matching  the  inpatient  and  outpatient  care  records  allows  a  complete  view  of 
patient  care  (Romano  &  Luft,  1992). 

Partnering  with  a  private  managed  care  organization  with  a  Medicare  product 
offers  additional  advantages.  The  managed  care  organization's  database  captured  all 
ICD-9  codes  for  each  patient.  When  exchanging  information  directly  with  Medicare, 
ICD-9  codes  are  limited  to  5  diagnoses  (Steinberg  et  al.,  1990).  This  can  lead  to  lost 
information  when  using  a  database  for  research  purposes.  It  has  been  shown  that  9  or 
more  ICD-9  codes  reduce  the  problem  of  under-representation  brought  about  by  this 
truncation  (Romano  &  Mark,  1994).  The  managed  care  database  did  not  limit  the 
number  of  diagnoses  or  procedures  codes  collected. 
Data  Fields 

The  data  were  received  from  the  managed  care  organization  as  two  separate 
databases,  one  for  medical  claims  and  the  second  containing  pharmacy  claims.  Each 
database  contained  the  identical  blinded  patient  identifier  that  could  not  be  traced  back  to 
the  individual  patient. 

The  following  data  fields  were  captured  from  the  medical  claims  database. 

•     Unique  patient  identification:  Data  were  stored  in  a  patient  specific  manner.  For 
purposes  of  the  study,  any  means  by  which  an  individual  patient  could  be 
identified  was  eliminated.  The  managed  care  organization  has  a  unique 
identification  code  for  each  patient.  This  code  was  used  to  create  a  new 
identification  number  to  eliminate  issues  with  confidentiality.  The  algorithm  for 
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coding  was  kept  at  the  managed  care  organization  and  was  not  available  to  the 
primary  researcher  (RJF). 

•  Coverage  period:  Each  patient  had  enrollment  dates  for  insurance  coverage. 
Patients  enrolled  for  at  least  12  months  during  the  study  period  were  included. 

•  Gender.  Patient  gender  was  available  in  the  database  as  male  and  female. 

•  Patient  age:  Each  patient's  birth  date  and  chronological  age  was  captured. 

•  Health  care  encounters:  Patient  health  care  encounters  were  coded  based  the  care 
site,  diagnosis,  and  the  services/products  rendered  to  the  patient.  These  were  in 
the  form  of  ICD-9  and  CPT  codes. 

•  Health  care  costs:  The  managed  care  organization  database  recorded  the  amount 
paid  for  each  specific  health  care  encounter  or  procedure  performed  from  the 
payer's  perspective.  A  separate  field  records  any  co-pays  or  deductibles  paid  by 
the  patient. 

•  Date  of  service:  The  database  recorded  the  date  (beginning  and  end)  of  each 
encounter. 

The  following  data  fields  were  captured  from  the  pharmacy  claims  database. 

•  Unique  patient  identification:  As  mentioned,  data  are  stored  in  a  patient  specific 
manner.  A  unique  identification  code  for  each  patient  was  provided,  which 
matched  that  in  the  medical  claims  database.  This  field  was  used  to  merge  the 
databases. 

•  Drug  identification:  National  Drug  Code  (NDC)  numbers  were  used  to  identify 
drug  use  for  all  definitions.  An  additional  field  records  the  American  Hospital 
Formulary  Service  (AHFS)  classification  for  each  medication.  The  AHFS  code 
was  useful  for  identifying  cardiovascular  drug  use. 

•  Prescriber  identification:  Each  prescriber  was  assigned  a  unique  code  which  was 
recorded  for  each  new  prescription. 


• 


Fill  date:  Each  prescription  had  the  date  filled  in  the  database.  There  was  a  date 
for  both  new  prescriptions  and  each  refill  date. 


•     Cost:  The  cost  of  the  medication  was  recorded  for  each  prescription.  The  cost 
was  separated  into  direct  expense  to  the  managed  care  organization  and  any 
patient  co-pays  or  deductibles. 
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Study  Variables 

Preventable  drug-related  morbidity 

The  primary  variable  of  interest  was  the  presence  of  preventable  drug-related 
morbidity.  The  measurement  of  preventable  drug-related  morbidity  involved  the 
identification  of  a  pattern  of  care  and  the  corresponding  outcome.  The  pattern  of  care  and 
outcome  were  identified  independently.  Patients  with  both  the  pattern  of  care  and  the 
outcome  for  each  consensus-approved  definition  were  counted  as  a  PRDM  event. 
Risk  factors 

As  discussed  in  chapter  2,  several  factors  are  related  to  the  presence  of  PDRM. 
These  were  used  as  confirmatory  risk  factors  in  the  study.  Each  risk  factor  was 
operationalized  as  follows: 

•  The  number  of  different  medications  taken  during  the  4  months  preceding  the 
PDRM  event  were  counted  and  operationalized  as  a  dichotomous  variable. 
Patients  receiving  6  or  more  medications  received  a  1 ,  zero  (0)  otherwise. 

•  A  patient  with  4  or  more  unique  prescribers  was  classified  as  a  1 ,  while  a 
patient  with  3  or  less  was  assigned  a  value  of  zero  (0). 

•  Patients  receiving  cardiovascular  drugs  were  classified  as  a  1 ,  zero  (0) 
otherwise. 

•  For  purposes  of  this  study,  females  were  assigned  a  value  of  1 ,  males  zero  (0). 

•  The  number  of  diagnoses  was  operationalized  a  dichotomous  variable.  A 
patient  with  4  or  more  diagnoses  was  classified  as  a  1,  zero  (0)  otherwise. 

•  Age  was  classified  as  dichotomous  variables  with  patients  over  85  (oldest  of 
the  old)  classified  as  a  1,  zero  (0)  otherwise. 

Health  care  resource  use  and  costs 

Health  care  resources  and  costs  were  operationalized  as  follows: 

•  The  number  of  hospitalizations  is  equal  to  the  number  of  individual 
hospitalizations  during  the  study  period.  The  cost  of  hospitalization  was 
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recorded  as  those  incurred  by  the  managed  care  organization  (MCO)  and  any 
patient  co-pays  or  deductibles. 

•  The  number  of  emergency  room  (ER)  visits  was  equal  to  the  number  of  times 
the  patient  visited  the  ER  during  the  study  period.  Once  again,  cost  was 
recorded  as  that  paid  by  the  MCO  and  any  patient  co-pays  or  deductibles. 

•  The  number  of  physician  office  visits  was  recorded  during  the  study  period. 
The  cost  was  that  incurred  by  the  MCO  and  any  co-pays  or  deductibles 
covered  by  the  patient. 

•  Medications  were  recorded  as  the  number  of  unique  prescription  drugs  a 
patient  received  during  the  4  months  preceding  their  PDRM  event  (if 
applicable).  Patients  without  an  instance  of  PDRM  had  the  number  of 
medications  recorded  during  the  first  four  months  of  enrollment.  The  total 
cost  is  that  incurred  by  MCO  and  any  co-pays  or  deductibles  covered  by  the 
patient  over  the  study  period. 

•  The  total  cost  of  care  is  the  sum  of  the  direct  medical  expenses  covered  by  the 
MCO  and  the  patient. 

Phase  IV:  Root  Cause  Analysis 

An  expert  panel  was  convened  to  determine  if  the  measure  of  preventable  drug- 
related  morbidity  could  be  used  in  a  quality  assessment  and  improvement  framework.  A 
root  cause  analysis  was  conducted  to  determine  if  practitioners  could  reason  the 
underlying  causes  of  quality  problems  for  the  ten  most  common  instances  of  PDRM. 

The  expert  panel  conducting  the  root  cause  analysis  consisted  of  4  physicians  and 
2  pharmacists  (Appendix  C).  The  ten  most  commonly  detected  PDRM  events  in  the 
patient  population  were  presented  to  the  panel.  The  panel  was  also  given  introductory 
information  about  the  study  objectives  and  the  process  of  root  cause  analysis.  The  panel 
was  asked  to  develop  cause-and-effect  diagrams  for  any  of  the  ten  most  common 
consensus-approved  definitions  of  preventable  drug-related  morbidity. 

A  quality  improvement  consultant  from  the  managed  care  organization  facilitated 
the  root  cause  analysis.  An  outside  consultant  was  employed  to  observe  group 
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interactions  that  took  place  during  the  root  cause  analysis.  In  particular,  the  consultant 

was  asked  to  observe  the  consensus  building  during  the  meeting. 

Statistical  Analysis 

Preventable  Drug-Related  Morbidity 

Preventable  drug-related  morbidity  was  the  primary  measure  of  interest.  The  rate 
of  preventable  drug-related  morbidity  in  our  population  was  equal  to  the  number  of 
patients  with  at  least  one  instance  of  preventable  drug-related  morbidity,  divided  by  the 
total  number  of  patients  eligible  for  the  study.  Because  the  study  viewed  up  to  18  months 
of  data,  the  number  of  patient  years  represented  by  the  population  was  calculated.  The 
prevalence  of  preventable  drug-related  morbidity  was  equal  to  the  number  of  patients 
with  PDRM  events  detected  divided  by  the  number  of  patient  years. 
Factors  Associated  With  Validity 

Variables  related  to  PDRM  from  previous  studies  served  as  evidence  of  validity 
of  the  definitions.  Eleven  variables  were  studied  as  confirmatory  hypotheses.  The 
variables  were  broken  down  into  risk  factors  and  health  care  resource  use. 
Risk  factors  as  confirmatory  hypotheses 

The  following  are  confirmatory  hypotheses  for  the  risk  factors: 

1 .  Females  will  tend  to  experience  preventable  drug-related  morbidity  more  than 
males. 

2.  Patients  receiving  6  or  more  medications  will  experience  preventable  drug- 
related  morbidity  more  often  than  those  receiving  less  than  6  medications. 

3.  Patients  receiving  cardiovascular  medications  will  have  a  higher  rate  of 
preventable  drug-related  morbidity  than  those  who  are  not. 

4.  Patients  with  4  or  more  prescribers  will  experience  preventable  drug-related 
morbidity  more  than  those  with  less  than  4  prescribers. 
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5.  Patients  with  4  or  more  diagnoses  will  experience  preventable  drug-related 
morbidity  more  often  than  those  with  less  than  4  diagnoses. 

6.  Patients  85  and  older  will  experience  preventable  drug-related  morbidity  more 
often  than  those  under  85. 

The  presence  of  risk  factors  was  assessed  using  a  logistic  regression  model. 
Logistic  regression  has  been  used  in  previous  studies  to  examine  adverse  drug  events  and 
risk  factors  (Bates  et  al.,  1999;  Escobedo  &  Zack,  1996;  MacKinnon,  1999;  Seeger  et  al., 
1 998).  Both  bivariate  and  multivariate  models  were  used  in  the  study. 

In  the  bivariate  analysis,  each  risk  factor  was  examined  as  an  independent 
variable  with  the  presence  of  PDRM  as  the  dependent  variable.  A  multiple  logistic 
regression  model  was  used  where  all  risk  factors  shown  to  be  significant  in  the  bivariate 
analysis  were  entered  simultaneously  to  determine  multivariate  significance. 

Significance  in  logistic  regression  models  was  determined  by  those  variables  with 
an  odds  ratio  of  greater  than  one  or  less  than  one  at  the  p  =  .05  level.  Positive 
associations  with  the  presence  of  preventable  drug-related  morbidity  are  those  variables 
with  an  odds  ratio  greater  than  one.  Those  variables  with  an  odds  ratio  less  than  one  were 
considered  negatively  associated  with  preventable  drug-related  morbidity. 
Resource  use  as  confirmatory  hypotheses 

As  described  previously,  patients  who  experience  preventable  drug-related 
morbidity  are  also  likely  to  use  more  health  care  resources.  The  following  are 
confirmatory  hypotheses  for  resource  use: 

7.  Patients  with  preventable  drug-related  morbidity  will  tend  to  have  more 
emergency  department  visits  than  those  without. 

8.  Patients  with  preventable  drug-related  morbidity  will  tend  to  have  more  hospital 
admissions  than  those  without. 
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9.  Patients  with  preventable  drug-related  morbidity  will  tend  to  have  more  doctor 
office  visits  than  those  without. 

10.  Patients  with  preventable  drug-related  morbidity  will  tend  to  cost  more  in  total 
direct  health  care  dollars  than  those  without. 

1 1 .  Patients  with  preventable  drug-related  morbidity  will  tend  to  have  higher 
medication  costs  than  those  without. 

To  test  these  confirmatory  hypotheses,  a  bivariate  logistic  regression  model  was 
used.  All  tests  for  significance  are  at  the  p  =  .05  level. 

Pattern  of  Care  and  Outcome  Analysis 

The  measurement  of  preventable  drug-related  morbidity  involved  identification  of 
a  pattern  of  care  and  outcome.  Linking  the  pattern  of  care  with  the  outcome  in  patients 
adds  to  the  validity  of  the  explicit  definitions.  By  identifying  the  pattern  of  care 
independent  of  the  outcome,  additional  analysis  was  possible  for  each  definition.  For 
each  definition,  patients  could  be  classified  into  one  of  four  categories: 

•  The  patient  had  both  the  pattern  of  care  and  the  corresponding  outcome,  and 
thus  preventable  drug-related  morbidity; 

•  The  patient  had  only  the  pattern  of  care  present  without  the  corresponding 
outcome; 

•  The  patient  had  only  the  outcome  detected;  or 

•  The  patient  had  neither  the  pattern  of  care  or  outcome  detected. 

Patients  with  evidence  of  PDRM  and  those  with  the  only  pattern  of  care  were  of 
most  interest  for  the  studies  purpose.  Those  patients  with  only  the  outcome  present  will 
be  discussed  briefly. 
Days  Between  the  Pattern  of  Care  and  Outcome 

The  pattern  of  care  and  corresponding  outcome  identified  had  the  date  of 
occurrence  recorded.  The  number  of  days  between  the  pattern  of  care  and  corresponding 
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outcome  was  calculated.  This  measure  is  referred  to  as  the  Number  of  Days  from  Pattern 

of  care  to  Outcome  (NDPO).  Each  consensus-approved  definition  with  more  than  one 

instance  of  preventable  drug-related  morbidity  detected  in  the  population  had  a  mean  and 

standard  deviation  for  NDPO  calculated. 

Patients  with  only  the  pattern  of  care  detected  but  not  the  adverse  outcome  also 
had  the  date  of  occurrence  attached.  A  calculation  was  made  to  determine  the  number  of 
days  between  the  pattern  of  care  and  the  end  of  the  study  (September  30,  1999).  This 
measure  was  useful  because  the  pattern  of  care  may  not  have  had  adequate  time  to 
culminate  in  the  adverse  outcome.  To  adjust  for  this  bias,  a  cut-off  point  was  sought  to 
estimate  the  number  of  days,  for  each  definition  with  1 0  or  more  PDRM  instances,  where 
the  pattern  of  care  would  be  expected  to  result  in  an  adverse  outcome.  A  cut-off  point 
was  determined  to  be  the  number  of  days  where  95%  of  the  patterns  of  care  would  be 
expected  to  result  in  the  corresponding  outcome. 

The  observed  distribution  of  the  NDPO  for  each  definition  with  10  or  more 
instances  was  examined  using  the  normal  distribution,  the  exponential  distribution  and 
the  gamma  distribution.  None  of  the  observed  distributions  appeared  to  fit  the  normal 
distribution.  The  gamma  distribution  did  not  prove  useful.  The  two  methods  chosen 
were  linear  interpolation  of  observed  data  and  the  cumulative  exponential  distribution. 
Linear  interpolation  estimated  where  the  95%  score  occurred  in  the  population.  The 
exponential  distribution  used  the  cumulative  function  to  determine  the  95%  cut-off  point. 
Patients  with  the  pattern  of  care  only  where  the  number  of  days  to  the  end  of  the  study 
was  longer  than  the  cut-off  point  were  included  in  additional  calculations  of  the 
association  between  the  pattern  of  care  and  the  outcome. 
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Process-To-Outcome  Value 

A  Process-To-Outcome  Value  (PTOV)  for  each  definition  with  at  least  ten 
instances  was  calculated.  The  PTOV  represents  the  proportion  of  patients  with  the 
pattern  of  care  (after  adjustment)  that  also  had  the  corresponding  outcome.  A  high  PTOV 
value  signifies  a  strong  association  between  the  pattern  of  care  and  the  corresponding 
outcome.  The  pattern  of  care  criteria  may  be  valid  if  it  can  be  demonstrated  that  what  is 
classified  as  poor  process  by  the  assessment  method  is  associated  with  poor  outcomes 
(Ashtonetal.,  1999). 


CHAPTER  4 
RESULTS 

Overview 

The  results  of  the  study  will  be  presented  in  four  sections.  The  Geriatric 
Medicine  Expert  Panel  and  the  consensus-approved  definitions  of  preventable  drug- 
related  morbidity  (PDRM)  will  be  presented  first.  The  second  section  will  give  the 
results  of  the  operationalization  of  the  definitions.  Thirdly,  the  measurement  of  PDRM 
and  associated  variables  in  the  study  population  will  be  detailed.  Finally,  the  root  cause 
analysis  will  be  presented  as  a  demonstration  of  the  use  of  a  PDRM  measure  in  quality 
assessment  and  improvement. 

Geriatric  Medicine  Expert  Panel 

The  Delphi  technique  was  used  to  develop  consensus  within  the  Geriatric 
Medicine  Expert  Panel  on  definitions  of  preventable  drug-related  morbidity  (PDRM). 
The  initial  survey  listed  55  explicit  definitions  of  preventable  drug-related  morbidity:  52 
definitions  adopted  from  MacKinnon's  (1999)  study  and  three  (3)  definitions  that  had 
been  rejected  by  MacKinnon's  (1999)  expert  panel.  Consensus  of  the  panel  was  obtained 
after  three  rounds.  Appendix  D  contains  the  final  list  of  49  consensus-approved 
definitions  of  preventable  drug-related  morbidity:  44  adopted  from  MacKinnon's  (1999) 
study,  2  that  had  been  rejected  by  MacKinnon's  (1999)  panel,  and  3  new  scenarios 
suggested  by  individual  panel  members  and  accepted  by  the  group. 
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Mnemonics 

A  series  of  mnemonics  have  been  created  to  facilitate  the  presentation  and 
discussion  of  the  results.  Appendix  E  lists  the  full  mnemonic  for  each  definition.  In 
Appendix  E  the  mnemonics  are  listed  with  the  outcome  first,  followed  by  a  forward  slash, 
then  the  pattern  of  care.  If  there  is  more  then  one  component  in  the  pattern  of  care,  each 
is  separated  by  a  dash  (outcome/pattern l-pattern2).  Appendix  F  contains  an  alphabetical 
list  of  the  mnemonic  components.  The  definition  number  and  the  mnemonic  will  be  used 
most  often  when  presenting  and  discussing  the  results. 
MacKinnon's  Definitions  Accepted  by  the  Panel 

The  panel  accepted  44  of  the  52  definitions  taken  from  MacKinnon's  (1999) 
study.  Table  4.1  lists  the  eight  definitions  rejected.  Two  of  the  definitions  rejected  by 
the  panel  relied  on  measuring  glycosylated  hemoglobin  (HbA|C)  levels  to  prevent  acute 
glycemic  events  (either  hyperglycemia  or  hypoglycemia).  In  both  cases,  the  panel  felt 
that  HbAic  levels  were  not  good  predictors  of  acute  events.  Panel  members  cited 
literature  and  argued  that  poor  blood  glucose  control  may  not  be  reflected  in  a  patient's 
glycosylated  hemoglobin  (HbAic)  level  in  time  to  prevent  an  acute  attack.  In  fact,  it  takes 
1-6  weeks  for  poor  blood  glucose  control  to  be  reflected  in  the  patient's  HbAic  level 
(DiPiro  et  al.,  1999;  Gebhart  et  al.,  1991).  Thus,  HbAic  levels  are  more  reflective  of 
chronic  blood  glucose  control  and  not  a  good  predictor  of  acute  events.  Blood  glucose 
monitoring  is  still  recommended  for  the  detection  and  avoidance  of  acute  events  (DiPiro 
et  al.,  1999).  Panel  members  recommended  that  fingersticks  would  be  an  acceptable 
pattern  of  care  for  the  scenario.  However,  daily  blood  glucose  monitoring  could  not  be 
measured  in  the  existing  administrative  database.  Thus,  the  recommended  scenario  was 
not  included. 
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Two  other  definitions  rejected  by  the  panel  had  a  fall  with  a  broken  bone  as  the 

outcome  linked  with  a  pattern  of  care  that  involved  the  use  of  either  antipsychotic  or 

anticholinergic  drugs.  In  both  cases,  the  panel  did  not  feel  the  association  between  the 

drug  use  and  the  adverse  outcome  was  strong  enough  to  accept  the  proposed  definitions. 

The  use  of  beta-blockers  in  patients  with  a  history  of  congestive  heart  failure  was 
also  rejected  as  an  instance  of  PDRM  by  the  panel.  Between  the  time  MacKinnon's 
(1999)  panel  developed  these  definitions  and  the  current  panel's  work,  evidence  emerged 
to  accept,  and  even  promote,  this  pattern  of  care  (Cohn,  2000;  DiPiro  et  al.,  1999; 
Tsuyuki  et  al.,  2000).  The  panel  cited  this  new  evidence  in  support  of  their  rejection  of 
the  scenario. 

The  panel  had  similar  reasons  for  rejecting  the  other  definitions.  The  panel 
rejected  these  events  due  to  the  lack  of  a  strong  direct  correlation  between  the  adverse 
outcome  and  the  pattern  of  care,  or  the  need  to  have  a  pattern  of  drug  use  for  a  patient 
population  (such  as  beta-blockers  in  diabetic  patients). 
MacKinnon's  Rejected  Definitions 

As  a  check  between  panels,  three  definitions  rejected  by  MacKinnon  (1999)  were 
inserted  into  the  first  survey.  By  the  end  of  the  third  survey,  two  of  the  three  definitions 
remained.  The  one  definition  rejected  by  both  panels  was: 

Outcome:  Fall  and/or  hip  fracture  and/or  bone  break 

Pattern  of  care:  Use  of  a  nitrate  (e.g.,  isosorbide,  etc.) 
Panel  members  rejected  this  scenario  based  on  the  weak  correlation  between  the  use  of 
nitrates  and  a  patient  fall,  with  or  without  a  broken  bone.  There  are  several  variables  that 
could  be  responsible  for  a  patient  fall  (Gales  &  Menard,  1995). 
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Table  4.1:  Original  Scenarios  Rejected  by  the  Panel 


Clinical  scenario 


Outcome:  ER  visit/hospitalization  due  to  hyperglycemia 

Pattern  of  Care:      1 .    Use  of  an  oral  hypoglycemic  agent  (e.g.,  chlorpropamide,  etc.) 

2.    Hemoglobin  Ale  level  not  done  at  least  every  6  months 

Outcome:  ER  visit/hospitalization  due  to  hypoglycemia  or  hyperglycemia 

Pattern  of  Care :      1 .    Use  of  insulin 

2.    Hemoglobin  Ale  level  not  done  at  least  every  6  months 

Outcome:  Fall  and/or  hip  fracture  and/or  bone  break 

Pattern  of  Care:      Use  of  an  antipsychotic  (e.g.,  thioridazine,  haloperidol, 

chlorpromazine,  etc.) 

Outcome:  Fall  and/or  hip  fracture  and/or  other  bone  fracture  and/or  bone  break 

Pattern  of  Care:      Use  of  an  anticholinergic  agent 


ER  visit/hospitalization  due  to  systolic  heart  failure 

1 .  History/diagnosis  of  systolic  heart  failure 

2.  Use  of  beta-adrenergic  blocking  agent  (e.g.,  propranolol,  nadolol, 
etc.) 

ER  visit/hospitalization  due  to  extreme  hypoglycemia 

1.  History/diagnosis  of  diabetes 

2.  Use  of  beta-adrenergic  blocking  agent  (e.g.,  propranolol,  nadolol, 
etc.) 

ER  visit/hospitalization  due  to  congestive  heart  failure 

1 .  History/diagnosis  of  congestive  heart  failure 
2.    Use  of  a  calcium  channel  blocker  (e.g.,  diltiazem,  etc.) 

Outcome:  Gastritis  and/or  upper  GI  bleed  and/or  GI  perforation  and/or  GI  ulcer 

and  anemia 
Pattern  of  Care:      1 .    NSAID  use  for  at  least  1  month 

2.  No  concurrent  use  of  a  cytoprotective  agent  (misoprostol) 

3.  Hemoglobin/hematocrit/CBC  not  done  within  30  days  of  the  start 
of  therapy  or  not  done  at  least  every  three  months  thereafter. 

NOTE:  Scenarios  were  taken  from  MacKinnon's  (1999)  study 


Outcome: 
Pattern  of  Care: 


Outcome: 
Pattern  of  Care: 


Outcome: 
Pattern  of  Care: 
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The  two  definitions  rejected  by  MacKinnon's  (1999)  panel  that  were  accepted  by 

the  current  panel  included  definition  3 1  (Dig/Dig-BUNCr-Druglvl)  and  40  (ARnF/Allp- 
BUNCr).  Both  definitions  have  support  in  the  literature  as  an  instance  of  PDRM  (Abad- 
Santos  et  al.,  2000;  Kumar  et  al.,  1996). 
Delphi  Panel  Survey  Results 

Three  rounds  of  surveys  were  needed  to  attain  consensus  within  the  Delphi  panel. 
The  panel's  level  of  agreement  improved  with  each  survey  (see  Table  4.2).  The 
percentage  of  all  definitions  having  83%  or  more  of  the  panel  voting  in  agreement  (i.e., 
voting  that  a  scenario  was  an  instance  of  PDRM)  increased  from  63.6%  in  round  1  to 
87.8%  in  round  3.  The  percentage  of  definitions  with  50-71%  affirmative  votes  and  those 
with  less  than  50%  (which  were  rejected)  declined  with  each  survey. 


Table  4.2:  PDRM  Definitions  Receiving  Percent  of  Agreement  Votes 


Panel  members  agreeing 

Round  1 

Round  2bc 

Round  3 

83%  or  more 

63.6%  (35)a 

69.8%  (37) 

87.8%  (43) 

50-71% 

27.3%  (15) 

22.6%  (12) 

12.2%  (6) 

Less  than  50% 

9.1%  (5) 

7.6%  (4) 

0.0%  (0) 

"Percent  of  proposed  definitions  receiving  the  level  of  agreement  from  panel  members 
l,One  panel  member  changed  between  rounds  1  and  2 
cThere  were  6  panel  members  included  in  round  1,  and  7  for  rounds  2  and  3. 
NOTE:  PDRM:  preventable  drug-related  morbidity 

Figure  4.1  illustrates  the  level  of  agreement  after  the  third  round  by  showing  the 
number  of  physicians  that  felt  a  scenario  was  an  instance  of  preventable  drug-related 
morbidity.  Out  of  the  49  definitions,  26  had  all  seven  panel  members  agreeing  it  was  an 
instance  of  PDRM,  17  had  six  out  of  seven,  and  6  had  five  out  of  seven.  No  definition 
received  less  than  five  members  agreeing  it  was  an  instance  of  preventable  drug-related 
morbidity.  This  level  of  agreement  was  judged  sufficient  to  end  the  Delphi  process  after 
the  third  survey  and  accept  49  scenarios  as  explicit  definitions  of  PDRM. 
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NOTE:  No  definition  received  less  than  5  votes  in  round  3 

Figure  4. 1 :  Number  of  Physicians  Agreeing  a  Scenario  is  an  Instance  of  PDRM 


Individual  Panel  Member  Voting  Results 

The  voting  pattern  for  each  panel  member  demonstrated  change  during  the  Delphi 

process.  Table  4.3  lists  each  individual  physician's  percent  of  scenarios  he  felt  was  an 
instance  of  PDRM  in  each  round. 

In  round  one  of  the  Delphi  process,  the  percent  of  scenarios  accepted  by  each 
physician  ranged  from  54.5%-90.9%,  with  an  average  of  76.4%.  Round  2  showed  an 
improvement,  with  a  range  of  71 .7%- 1 00%,  and  an  average  of  87.8%.  The  final  round 
demonstrated  slight  improvement  with  a  range  and  average  of  8 1.6%- 100%  and  92.7%, 
respectively. 

The  percent  of  definitions  each  panel  member  agreed  was  an  instance  of 
preventable  drug-related  morbidity  increased  with  each  survey.  The  range  of  percentage 
point  increase  per  physician  was  4.5-27. 1 ,  with  an  average  increase  of  1 5.2.  Thus,  all 
panel  members  progressed  in  their  level  of  agreement  with  each  round. 


57 


Table  4.3:  Individual 

^anel  Member  Percent  Agreement 

Panel  member 

Round  1 

Round  2 

Round  3 

1 

87.3%  (48) 

92.4%  (49) 

91.8%  (45) 

2 

87.3%  (48) 

98.1%  (52) 

100.0%  (49) 

5 

90.9%  (50) 

100.0%  (53) 

100.0%  (49) 

4 

N/Aa 

83.0%  (44) 

93.8%  (46) 

5 

72.7%  (40) 

86.7%  (46) 

93.8%  (46) 

6 

54.5%  (30) 

71.7%  (38) 

81.6%  (40) 

7 

65.5%  (36) 

83.0%  (44) 

87.7%  (43) 

"Panel  member  number  4  was  changed  between  rounds  1  and  2 


Disqualification  of  Panel  Member 

After  round  1 ,  the  surveys  and  comments  were  reviewed  and  tabulated.  Based  on 
written  comments,  it  was  evident  that  one  panel  member  did  not  agree  with  the  goals  of 
the  study.  Because  of  this  member's  interpretation  of  the  study,  all  panel  members 
received  retraining  on  the  goals  and  process  of  the  study. 

The  retraining  of  the  panel  took  place  in  one-on-one  meetings  with  the  principal 
investigator  (RJF).  The  meetings  followed  a  structured  agenda  that  reviewed  the  goals  of 
the  study,  the  definitions  and  framework  of  the  survey,  the  instructions  for  completing  the 
survey,  and  open  time  for  questions. 

Based  on  the  first  survey,  the  individual  meeting,  and  the  response  on  the  second 
survey,  it  became  necessary  to  disqualify  the  previously  mentioned  panel  member. 
During  the  one-on-one  meeting,  it  was  clear  that  this  particular  panel  member  would  not 
support  the  goal  of  the  project.  During  that  meeting  he  stated,  "I  don't  care  what  you  say 
these  definitions  will  be  used  for,  I  know  that  [MCO  name]  will  use  them  as  practice 
standards  and  tell  me  I  am  not  doing  things  right." 

After  the  disqualification  of  the  panel  member,  a  replacement  was  selected.  This 
panel  member  received  the  same  instructions  as  other  panel  members  (both  written  and 
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oral).  Sitting  panel  members  were  not  informed  of  the  disqualification  or  the  new  panel 

member.  The  new  panel  member  began  by  completing  the  second  survey. 

Operationalization  of  Definitions 

The  consensus-approved  definitions  of  preventable  drug-related  morbidity  were 

operationalized  for  application  to  an  administrative  database.  The  pattern  of  care  and 

outcome  for  each  definition  were  coded  separately.  Diagnosis  and  medical-related 

procedures  were  coded  using  International  Classification  of  Diseases-91  edition  (ICD-9) 

and  Current  Procedural  Terminology  (CPT)  codes.  National  Drug  Codes  (NDC)  were 

used  to  identify  medication  use. 

Medical  Event  Coding 

Two  medical  record  coders  independently  selected  all  possible  ICD-9  and  CPT 

codes  for  each  pattern  of  care  and  corresponding  adverse  outcome.  The  two  sets  of  codes 
were  combined  and  any  code  selected  by  at  least  one  coder  was  included  as  a  possible 
code  for  each  definition. 

The  medical  record  coders  identified  the  same  codes  approximately  60%  of  the 
time  (standard  deviation  of  27.1).  The  range  of  identical  codes  selected  by  both  coders 
varied  from  0-100%.  The  extreme  values  were  more  likely  to  occur  when  the  coders 
identified  fewer  total  codes  per  definition. 

Overall,  the  medical  record  coders  identified  a  total  of  6,536  unique  codes  for  the 
49  definitions.  The  number  of  codes  identified  for  each  definition  ranged  from  1  to  843. 
The  average  number  of  codes  identified  per  definition  was  133.4  with  a  median  value  of 
3 1 ,  reflecting  a  skewed  distribution  to  the  right. 
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Physician  Review  of  Codes 

The  codes  identified  by  the  medical  record  coders  were  given  to  a  physician  for 
clinical  judgment  as  to  whether  each  code  was  consistent  with  the  consensus-approved 
definition.  The  physician  reviewer  judged  4,497  of  the  6,536  codes  (68.8%)  as  definitely 
consistent  with  the  definition,  747  (1 1.4%)  codes  as  possibly  consistent  with  the 
definition,  and  1,292  (19.8%)  of  the  codes  as  not  consistent  with  the  definition.  All  codes 
deemed  not  consistent  with  the  definition  were  eliminated.  The  percent  of  codes 
eliminated  per  definition  ranged  from  0-53.9%. 

Preventable  Drug-Related  Morbidity  Measurement 
Study  Population 

Overall,  there  were  1 1,71 1  patients  eligible  for  inclusion  in  the  study.  The 
majority  of  the  study  population  were  female  (57.3%).  The  average  age  of  participants 
was  73.8  years,  with  approximately  1 1%  of  the  study  population  85  years  or  older. 

Patients  were  covered  by  the  managed  care  organization  an  average  of  1 .32  years 
during  the  18-month  study.  Thus,  the  1 1,71 1  patients  accounted  for  15,458.5  patient 
years. 
Preventable  Drug-Related  Morbidity 

Of  the  1 1,71 1  patients,  966  (8.2%)  had  at  least  one  instance  of  PDRM,  i.e.,  the 
presence  of  a  code  for  the  adverse  outcome  and  code(s)  for  the  pattern  of  care  that  were 
consistent  with  one  of  the  operational  definitions  adopted  by  the  Delphi  panel  (see 
Appendix  D).  Because  patients  were  enrolled  an  average  of  1 .32  years,  the  prevalence 
per  year  was  6.25%.  Of  those  patients  with  a  PDRM  event  detected,  685  (70.9%)  had 
only  one  instance  of  PDRM,  199  (20.6%)  had  2,  and  82  (8.5%)  had  three  or  more 
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instances.  Appendix  G  lists  the  number  of  events  detected  in  the  patient  population  for 

each  consensus-approved  definition  of  preventable  drug-related  morbidity. 

A  total  of  1,359  total  events  were  detected  in  the  study  population.  Table  4.4  lists 
the  20  most  common  instances  of  preventable  drug-related  morbidity  detected.  The  five 
most  common  definitions  account  for  approximately  57%  of  PDRM  events  detected.  The 
ten  most  common  definitions  detected  79.6%  of  events.  Thus,  20%  of  the  definitions 
account  for  about  80%  of  preventable  drug-related  morbidity  detected  in  the  study 
population. 

The  study  population  did  not  exhibit  any  instances  of  PDRM  in  20  (approximately 
40%)  of  the  49  consensus-approved  definitions  of  preventable  drug-related  morbidity. 
Pattern  of  Care  and  Outcome  Measurement 

The  method  employed  to  measure  preventable  drug-related  morbidity  in  the 
patient  population  detected  occurrences  for  each  pattern  of  care  and  corresponding 
outcome  independently.  Appendix  H  lists  the  number  of  patients  who  had  only  the 
pattern  of  care,  only  the  outcome,  or  both  (thus  PDRM)  for  each  definition. 

Outcomes  were  detected  in  a  number  of  patients  without  the  corresponding 
pattern  of  care  present  (thus,  it  was  not  an  instance  of  PDRM  as  defined  by  the  panel). 
Definitions  4  (Hmr/Hep-  NoLab),  6  (Dep/Hx-Barb),  1 1  (Hmr/War-NSAID-INR),  and  38 
(Hmr/War-ABx-NoLab)  had  at  least  500  patients  with  the  outcome  present,  but  2  or 
fewer  patterns  of  care  detected.  Three  of  the  definitions  (4,  11,  and  38)  detected  bleeding 
as  a  common  adverse  outcome.  This  result  indicates  that  the  adverse  outcomes  were 
probably  not  caused  by  a  preventable  drug-related  problem.  Non-drug-related  causes  for 
adverse  events  in  medical  care  are  common  (Brennan  et  al.,  1991;  Leape  et  al.,  1991). 
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Table  4.4:  Twenty  Most  Common  PDRM  Events  Detected 

Definition3 

Mnemonicb 

Instances  of  PDRMC 

Percent  of  total  PDRMC 

45 

CHF/Hx-ACEI 

270 

19.90% 

33 

CHF/Hx-Dig 

184 

13.50% 

39 

GI/NSAID 

129 

9.50% 

22 

Hypoth/Thy-T4TSH 

103 

7.60% 

48 

Asth/Hx-Bdilat-ICort 

89 

6.50% 

19 

CHF/Hx-NSAID 

80 

5.90% 

5 

GI/Hx-NSAID 

63 

4.60% 

1 

Dep/Hx-Benzo 

56 

4.10% 

25 

COPD/Hx-BB 

54 

4.00% 

36 

MI/Hx-ASA-BB 

54 

4.00% 

20 

HypoK/Kwd-NoK-Elec 

44 

3.20% 

44 

FB/SedHyp 

42 

3.10% 

32 

GI/Hx-OCort 

38 

2.80% 

47 

CHF/Hx-AArhy 

37 

2.70% 

7 

Dep/Hx-Symp 

35 

2.60% 

16 

ARsF/COPD-Benzo 

19 

1.40% 

41 

Dep/Hx-BB 

15 

1.10% 

28 

HyprK/ACEI-EleCBC 

14 

1 .00% 

8 

Seiz/Antconv-DrugLvl 

10 

0.70% 

49 

FB/TCA 

8 

0.60% 

'Definition  number  matches  Appendix  D 
''Mnemonics  are  in  Appendix  E  and  F 
CPDRM:  preventable  drug-related  morbidity 
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There  were  several  definitions  for  which  a  large  number  of  patients  had  the 

pattern  of  care  detected  without  the  corresponding  outcome  present.  Definitions  1 0 

(Hyprth/Thy-T4TSH),  28  (HyprK/ACEI-EleCBC),  34  (ARnF/ACEI-BUNCr),  46 

(ARnF/NSAID-BUNCr),  and  49  (FB/TCA)  were  the  most  common  definitions  where 

patients  had  only  the  pattern  of  care  detected.  Four  of  those  definitions  (10,  28,  34,  and 

46)  had  some  form  of  laboratory  monitoring  requirement.  Two  definitions  (28  and  34) 

involve  the  use  of  ACE  inhibitors  with  laboratory  monitoring. 

There  were  nine  definitions  for  which  the  majority  of  the  patients  with  a  pattern  of 
care  present  also  had  the  corresponding  outcome.  Definitions  1  (Dep/Hx-Benzo),  5 
(GI/Hx-NSAID),  7  (Dep/Hx-Symp),  25  (COPD/Hx-BB),  32  (GI/Hx-OCort),  and  48 
(Asth/Hx-Bdilat-ICort)  are  examples.  These  definitions  will  be  discussed  later  as  they 
relate  to  the  Process-To-Outcome  Value. 
Days  Between  Pattern  of  Care  and  Outcome 

In  each  instance  of  preventable  drug-related  morbidity,  the  number  of  days 
between  the  pattern  of  care  and  the  corresponding  outcome  was  determined.  This  is 
referred  to  as  the  Number  of  Days  between  the  Pattern  of  care  and  Outcome  (NDPO)  (see 
Chapter  3).  The  average  NDPO  and  standard  deviation  for  all  definitions  with  at  least  10 
detected  instances  are  listed  in  Table  4.5. 

The  average  NDPO  per  definition  for  a  PDRM  event  to  occur  ranged  from  1 8.81- 
203.50  (overall  average  of  130.06).  Six  of  the  10  definitions  with  the  longest  average 
number  of  days  between  the  pattern  of  care  and  the  outcome  had  a  laboratory  monitoring 
requirement.  Conversely,  only  two  of  the  1 0  preventable  drug-related  morbidity  events 
with  the  shortest  average  NDPO  had  a  requirement  for  laboratory  monitoring.  Thus,  the 
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monitoring  of  laboratory  results  may  allow  the  practitioner  added  time  to  recognize  a 

potential  drug  therapy  problem  and  prevent  the  adverse  outcome  from  occurring. 

Adjusted  Davs  for  Preventable  Drug-Related  Morbidity 

Patients  with  only  the  pattern  of  care  had  the  number  of  days  between  its 
occurrence  and  the  end  of  the  study  period  (September  30, 1999)  calculated.  Due  to  the 
truncation  of  the  data  collection  period  at  18  months,  some  of  the  patterns  of  care  might 
have  resulted  in  an  adverse  outcome  if  adequate  time  had  been  allowed.  Thus,  the  data 
for  actual  PDRM  events  were  used  to  determine  a  95%  cut-off  point  where  most  patterns 
of  care  could  be  expected  to  result  in  the  corresponding  adverse  outcome.  Those  patterns 
of  care  within  the  95%  cut-off  limits  were  not  considered  in  the  calculation  of  the 
Process-To-Outcome  Value.  If  all  patterns  of  care  were  used,  regardless  of  time  that  had 
passed  since  its  appearance,  the  PTOV  would  be  biased  downward. 

The  distribution  of  the  NDPO  for  the  1 9  consensus-approved  definitions  with  ten 
or  more  instances  was  reviewed.  The  distributions  were  not  normally  distributed. 
Therefore,  mean  plus  1.645  standard  deviations  did  not  appear  to  be  an  appropriate 
estimate  of  the  95%  cumulative  frequency  cut-off  point.  Two  other  methods  were  used 
instead.  Linear  interpolation  of  the  observed  cumulative  frequency  distribution  was  used 
to  calculate  the  95%  cut-off  point  for  definitions  1  (Dep/Hx-Benzo),  5  (GI/Hx-NSAID),  7 
(Dep/Hx-Symp),  16  (ARsF/COPD-Benzo),  19  (CHF/Hx-NSAID),  20  (HypoK/Kwd- 
NoK-Elec),  25  (COPD/Hx-BB),  32  (GI/Hx-Ocort),  33  (CHF/Hx-Dig),  36  (MI/Hx-ASA- 
BB),  41  (Dep/Hx-BB),  45  (CHF/Hx-ACEI),  47  (CHF/Hx-AArhy)  and  48  (Asth/Hx- 
Bdilat-ICort).  Because  the  data  for  definitions  8  (Seiz/Antconv-DrugLvl),  22 
(Hypoth/Thy-T4TSH),  28  (HyprK/ACEI-EleCBC),  39  (GI/NSAID),  and  44  (FB/SedHyp) 
were  highly  right-skewed,  we  chose  to  use  the  cumulative  exponential  distribution. 


Table  4.5:  Average  Number  o 

04 

'Days  Between  Pattern  of  Care  and  Outcome  (NDPO) 

Definition  number3  and 
mnemonicb 

Number  of 

PDRMC  events 

detected 

Mean  number  of  days 

between  pattern  of 
care  and  the  outcome 

Standard 
deviation 

1  (Dep/Hx-Benzo) 

56 

139.93 

339.98 

5  (GI/Hx-NSAID) 

63 

83.35 

104.80 

7  (Dep/Hx-Symp) 

35 

107.57 

104.29 

8  (Seiz/Antconv-DrugLvl) 

10 

142.00 

74.61 

16(ARsF/COPD-Benzo) 

19 

133.84 

116.89 

17(AUR/Hx-Antic) 

4 

114.75 

130.34 

19(CHF/Hx-NSAID) 

80 

150.55 

122.99 

20  (HypoK/Kwd-NoK-Elec) 

44 

126.00 

112.93 

22  (Hypoth/Thy-T4TSH) 

103 

182.20 

118.66 

25  (COPD/Hx-BB) 

54 

115.03 

133.64 

28  (HyprK/ACEI-EleCBC) 

14 

199.50 

118.02 

32  (GI/Hx-Ocort) 

38 

88.95 

102.37 

33  (CHF/Hx-Dig) 

184 

150.65 

122.76 

35  (Hmr/War-INR) 

3 

69.33 

47.17 

36  (MI/Hx-ASA-BB) 

54 

18.81 

38.44 

39  (GI/NSAID) 

129 

171.62 

118.82 

41  (Dep/Hx-BB) 

15 

130.07 

113.97 

44  (FB/SedHyp) 

42 

192.38 

111.45 

45  (CHF/Hx-ACEI) 

270 

121.28 

127.75 

46  (ARnF/NSAID-BUNCr) 

2 

203.50 

207.18 

47  (CHF/Hx-AArhy) 

37 

131.35 

112.22 

48  (Asth/Hx-Bdilat-ICort) 

89 

161.03 

132.85 

49  (FB/TCA) 

8 

125.75 

84.60 

"Definition  number  matches  Appendix  D 
"'Mnemonic  matches  appendices  E  and  F 
LPDRM:  preventable  drug-related  morbidity 
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Appendix  I  compares  the  95%  cut-off  point  (in  days)  for  the  two  methods  chosen 

(linear  interpolation  of  the  observed  cumulative  frequency  distribution  and  the 

cumulative  exponential  distribution)  with  the  cut-off  point  if  the  data  were  assumed  to  be 

normally  distributed  (mean  plus  1.645  standard  deviations).  The  comparison  of  the  three 

estimated  cut-off  points  demonstrates  the  variation  due  to  the  distribution  of  the  data.  In 

14  of  the  19  definitions,  the  estimates  vary  by  more  than  10  percent.  Eleven  of  the  19 

estimates  demonstrated  a  difference  greater  than  20  percent. 

The  cumulative  exponential  distribution  and  the  linear  interpolation  yielded 
similar  results  for  several  definitions,  with  13  out  of  19  having  a  difference  of 
approximately  20%  or  less.  Eight  out  of  19  definitions  had  differences  less  than  10% 
when  comparing  the  two  methods. 
Patients  With  the  Pattern  of  Care  After  Adjustment 

There  were  6,604  patterns  of  care  detected  in  the  patient  population  for  the  19 
definitions  with  10  or  more  instances  of  PDRM.  Of  this  total,  1,336  had  the 
corresponding  outcome  present  (thus,  PDRM).  After  the  95%  cut-off  point  was  applied 
to  each  pattern  of  care,  4,138  patients  with  the  pattern  of  care  were  included  in  the  PTOV 
calculation.  Appendix  J  lists  the  number  of  patients  with  each  PDRM  event,  the  number 
with  only  the  pattern  of  care  using  no  cut-off  point,  and  the  number  of  patients  with  the 
pattern  of  care  after  the  95%  cut-off  adjustment  was  applied.  The  number  of  patients 
eliminated  who  only  had  the  pattern  of  care  ranged  from  0-675  per  definition,  with  an 
average  of  129.8. 
Process-To-Outcome  Value 

A  Process-To-Outcome  Value  (PTOV)  was  estimated  for  each  definition  with  1 0 
or  more  instances  of  PDRM  detected  in  the  study  population.  The  PTOV  reflects  the 
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proportion  of  patterns  of  care  (after  adjustment)  in  the  population  that  resulted  in  the 

corresponding  adverse  outcome. 

Table  4.6  lists  the  Process-To-Outcome  Value  for  each  definition  with  10  or  more 
instances  of  PDRM  detected  in  the  study  population.  The  initial  PTO V  is  based  on  all 
patients  with  the  pattern  of  care  prior  to  application  of  the  95%  the  cut-off  adjustment. 
The  adjusted  Process-To-Outcome  Value  is  based  on  the  patterns  of  care  remaining  after 
the  95%  cut-off  values  for  each  definition  were  applied. 

The  Process-To-Outcome  Value  for  definitions  with  1 0  or  more  instances  of 
PDRM  ranged  from  0.04-1 .00.  Nearly  half  (9  of  19)  of  the  definitions  had  a  PTOV 
greater  than  0.75.  These  definitions  demonstrate  a  high  association  between  the  pattern 
of  care  and  the  adverse  outcome  detected  in  the  database. 

There  were  five  definitions  where  the  Process-To-Outcome  Value  was  less  than 
0.25.  These  definitions  contain  a  pattern  of  care  that  appears  less  likely  to  result  in  an 
adverse  outcome. 

Confirmatory  Hypotheses 
Risk  Factors 

MacKinnon's  (1999)  study  identified  risk  factors  associated  with  the  presence  of 
PDRM.  These  risk  factors  were  used  as  confirmatory  hypotheses  in  my  study.  Each  risk 
factor  was  examined  as  an  independent  variable  in  a  logistic  regression  model.  All  risk 
factors  were  significantly  associated  with  the  presence  of  preventable  drug-related 
morbidity  (Table  4.7). 

All  risk  factors  were  entered  simultaneously  in  a  multivariate  logistic  regression 
model.  In  the  multivariate  model,  gender  and  cardiovascular  drugs  were  no  longer 
significantly  associated  with  the  presence  of  PDRM.  All  other  confirmatory  risk  factors 
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Table  4.6:  PTOV  Values  Basec 

on  Various  Cut-o 

T  Levels 

Definition3'" 

Initial  PTOVc 

Adjusted  PTOVd 

1  (Dep/Hx-Benzo) 

0.98 

1.00 

5  (GI/Hx-NSAID) 

0.98 

0.98 

7  (Dep/Hx-Symp) 

1.00 

1.00 

8  (Seiz/Antconv-DrugLvl) 

0.20 

0.22 

16(ARsF/COPD-Benzo) 

0.10 

0.26 

19(CHF/Hx-NSAID) 

0.09 

0.37 

20  (HypoK/Kwd-NoK-Elec) 

0.06 

0.11 

22  (Hypoth/Thy-T4TSH) 

0.37 

0.45 

25  (COPD/Hx-BB) 

0.98 

1.00 

28  (HyprK/ACEI-EleCBC) 

0.03 

0.04 

32  (GI/Hx-Ocort) 

0.97 

1.00 

33  (CHF/Hx-Dig) 

0.79 

0.95 

36  (MI/Hx-ASA-BB) 

0.55 

0.62 

39  (GI/NSAID) 

0.08 

0.12 

41  (Dep/Hx-BB) 

0.94 

1.00 

44  (FB/SedHyp) 

0.05 

0.06 

45  (CHF/Hx-ACEI) 

0.47 

0.75 

47  (CHF/Hx-AArhy) 

0.58 

0.86 

48  (Asth/Hx-Bdilat-ICort) 

1.00 

1.00 

Average  PTOV 

0.54 

0.62 

"Definition  number  matches  Appendix  D 

''Mnemonic  matches  appendices  E  and  F 

cBefore  cut-off  adjustment 

dAfter  patterns  of  care  eliminated  based  on  cut-off  point 


Table  4.7:  Confirmatory  Risk  Factors 

Factor 

Odds  ratio 

95%  CIa  for  odds  ratio 

Sig.B 

Gender 

1.22 

1.07-1.40 

<0.004 

Patients  over  85 

1.60 

1.33-1.92 

<0.0001 

Number  of  prescribers 

4.57 

3.98-5.24 

O.0001 

Number  of  drugs 

5.98 

5.21-6.85 

<0.0001 

Number  of  diagnoses 

5.24 

2.89-9.51 

<0.0001 

Cardiovascular  drugs 

6.74 

5.36-8.47 

<0.0001 

CI:  confidence  interval 
bSig:  significance  of  odds  ratio 
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were  significant.  Evidence  of  multicollinearity  among  the  variables  was  examined.  No 

two  independent  variables  had  a  correlation  above  0.3,  with  most  0.1  or  less. 
Resource  Utilization  and  Cost 

Resource  utilization  and  costs  were  examined  to  determine  if  patients  with 
preventable  drug-related  morbidity  tended  to  use  more  resources  and  cost  more  than 
patients  without  an  instance  of  PDRM.  Based  on  the  analysis,  all  categories  examined 
were  positively  associated  with  the  presence  of  preventable  drug-related  morbidity. 
Table  4.8  lists  the  average  direct  health  care  costs  for  patients  with  and  without  the 
presence  of  PDRM. 


Table  4.8:  Direct  Hea 

th  Care  Costs 

Patients  without  PDRM 

Patients  with  PDRM 

Significance 

Total  direct  costs 

$3,423 

$16,821 

<  0.0001 

Medical-related 
costs 

$2696 

$15,043 

<  0.0001 

Pharmacy-related 
costs 

$446 

$1,069 

<  0.0001 

NOTE:  PDRM:  preventable  drug-related  morbidity 

In  addition  to  direct  medical  expenses,  patients  with  preventable  drug-related 
morbidity  tended  to  have  more  hospital  admissions,  visit  the  emergency  room  more  often, 
go  to  the  physician's  office  more  often,  and  use  more  drugs  than  those  patients  with  no 
instance  of  PDRM  detected. 

Root  Cause  Analysis 

A  root  cause  analysis  was  conducted  to  determine  if  the  measure  of  preventable 
drug-related  morbidity  based  on  consensus-approved  definitions  could  be  used  to  identify 
systematic  problems  in  quality  of  the  medication  use  system.  An  expert  panel  of  six 
health  care  professionals  participated  in  the  root  cause  analysis  (Appendix  C). 
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A  quality  consultant  employed  by  the  managed  care  organization  facilitated  the 

meeting.  The  principal  investigator  (RJF),  a  faculty  member  from  the  University  of 

Florida,  and  a  group  interaction  observer  were  also  present  at  the  meeting.  The  principal 

investigator  (RJF)  introduced  the  topic  and  described  the  information  distributed  to  panel 

members.  At  that  point,  the  quality  consultant  facilitated  the  root  cause  analysis. 

The  primary  outcome  of  the  meeting  was  a  series  of  cause-and-effect  diagrams 
(appendices  K,  L,  M,  and  N).  These  diagrams  list  possible  root  causes  identified  by  the 
panel  for  several  consensus-approved  definitions  of  preventable  drug-related  morbidity. 
A  number  of  root  causes  were  identified  for  each  PDRM  definition  addressed  by  the 
panel.  Possible  root  causes  were  grouped  into  5  major  categories:  1)  the  provider,  2)  the 
insurer,  3)  the  patient,  4)  the  medication  use  process,  and  5)  other  causes. 

Dr.  Barbara  Brice  observed  group  dynamics  during  the  meeting  and  provided  a 
written  report  (Appendix  O).  While  consensus  was  not  obtained  due  to  time  constraints 
for  the  meeting,  the  group  interaction  and  results  demonstrate  the  use  of  a  PDRM 
measure  as  a  quality  improvement  tool. 


CHAPTER  5 
DISCUSSION 

Introduction 

Chapter  5  will  discuss  the  findings  and  implications  of  our  study.  First,  the 
consensus-approved  definitions  and  the  voting  patterns  of  the  Delphi  panel  will  be 
discussed,  followed  by  the  operationalization  of  the  definitions  and  the  measurement  of 
preventable  drug-related  morbidity.  The  root  cause  analysis  will  be  presented  as  it  relates 
to  the  use  of  the  measure.  Additionally,  the  validity  of  the  definitions  will  be  addressed. 
Finally,  concluding  remarks  will  be  offered  along  with  the  study's  limitations, 
significance  and  implications  for  future  research. 

Geriatric  Medicine  Expert  Panel 
Replication  of  MacKinnon's  Work 

The  Geriatric  Medicine  Expert  Panel  was  convened  to  gain  consensus  on  explicit 
definitions  of  preventable  drug-related  morbidity.  The  panel  used  MacKinnon's  (1999) 
work  as  a  starting  point.  MacKinnon  (1999)  used  literature  and  the  Delphi  technique  to 
define  52  explicit  scenarios  of  preventable  drug-related  morbidity.  Replicating 
MacKinnon's  (1999)  methodology  strengthens  validity  of  our  explicit  definitions  by 
using  a  different  expert  panel  to  come  to  an  independent  conclusion  about  their  relevance 
and  representativeness. 

The  current  study  used  the  Delphi  technique  with  three  rounds  to  attain 
consensus.  There  was  a  high  level  of  agreement  after  the  third  round,  with  43  out  of  49 
(87.8%)  definitions  having  either  all  7,  or  6  out  of  the  7  panel  members  agreeing  it  was 
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an  instance  of  PDRM.  No  definition  in  the  third  survey  received  less  than  71%  (5  out  of 

7)  of  the  panel  agreeing  it  was  an  instance  of  PDRM. 

MacKinnon's  (1999)  panel  required  two  rounds  to  gain  consensus.  The 
difference  in  the  number  of  rounds  needed  to  gain  consensus  could  be  due  to  the  fact  that 
all  MacKinnon's  (1999)  panel  members  practiced  in  the  same  location.  The  chief 
physician  at  this  location  also  took  an  aggressive  leadership  role  in  the  project.  The 
current  panel  had  more  diverse  practice  settings.  No  two  physicians  were  in  the  same 
group  practice  (although  three  were  faculty  members  at  the  same  university  but  practiced 
at  different  locations).  The  diversity  of  practice  sites  in  the  current  panel  demonstrates 
that  definitions  of  preventable  drug-related  morbidity  may  have  broad  agreement  among 
physicians. 
Consensus- Approved  Definitions 

The  Geriatric  Medicine  Expert  Panel  accepted  49  definitions  of  preventable  drug- 
related  morbidity  (Appendix  D),  including  44  of  the  52  definitions  from  MacKinnon's 
(1999)  study.  Panel  members  often  cited  literature  and  specific  criteria  to  support  their 
decision  to  either  accept  or  reject  a  specific  scenario.  Thus,  the  panel  demonstrated  an 
evidence-based  thought  process,  adding  to  the  validity  of  out  results  (Craig  et  al.,  2001; 
Rosenberg  &  Donald,  1995). 

The  panel  accepted  2  of  the  three  definitions  rejected  in  MacKinnon's  (1999) 
study.  This  result  could  call  into  question  the  validity  of  the  definitions.  However,  both 
definitions  had  very  specific  laboratory  monitoring  requirements.  MacKinnon's  (1999) 
panel  rejected  these  definitions  because  they  disagreed  with  the  length  of  time  specified 
for  laboratory  monitoring,  not  the  connection  of  pattern  of  care  to  outcome  (personal 


72 
communication  between  the  principal  investigator  (RJF)  and  MacKinnon.  2000).  Thus, 

acceptance  by  the  panel  is  not  extraordinary. 

The  panel  also  suggested  three  additional  definitions  that  were  ultimately 

accepted.  Other  definitions  were  suggested  that  were  either  rejected  by  the  panel  or  were 

not  presented  to  the  panel  because  I  could  not  operationalize  the  definitions. 

Evidence-Based  Medicine  and  the  Panel 

Evidence-based  medicine  has  been  defined  as  "the  process  of  systematically 
finding,  appraising  and  using  contemporaneous  research  findings  as  the  basis  for  clinical 
decisions"  (Rosenberg  &  Donald,  1995,  p.l  122).  MacKinnon  (1999)  used  the  literature 
to  develop  each  definition  of  preventable  drug-related  morbidity.  New  definitions  also 
had  to  be  supported  by  the  literature.  Thus,  each  definition  was  evidence-based. 

The  research  method  employed  by  MacKinnon  (1999)  and  the  current  study 
directed  panel  members  to  operate  in  a  manner  consistent  with  the  definition  of  evidence- 
based  medicine  offered  by  Rosenberg  and  Donald  (1995).  Panel  members  were  given 
definitions  that  were  developed  based  on  the  literature,  then  used  their  knowledge  of  the 
evidence  and  their  practice  patterns  to  accept  or  reject  the  definitions.  Thus,  there  is 
interplay  between  evidence  and  practice.  This  methodology  is  offered  in  contrast  to  the 
use  of  expert  opinion  with  no  direct  citation  of  evidence  for  support  (otherwise  referred  to 
as  implicit  judgement). 

Implicit  judgement  has  demonstrated  a  wide  variability  and  may  produce  invalid 
results  because  experts  do  not  agree.  Our  study  may  offer  a  baseline  for  expert 
agreement  when  presented  with  evidence. 
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Panel  Developed  Explicit  Definitions  Versus  Implicit  Judgement 

Broad  agreement  between  MacKinnon's  (1999)  panel  and  the  current  study  was 
demonstrated.  The  differences  that  exist  are  expected  when  relying  on  the  interplay 
between  evidence  and  clinical  judgement.  Differences  may  be  traced  to  new  evidence 
between  the  time  the  two  panels  convened  and  practice  pattern  variation.  The  results 
suggest  that  the  development  of  evidence-based,  explicit  definitions  for  PDRM  can  be 
accomplished  with  practicing  physicians. 

Explicit  definitions  are  offered  as  an  alternative  to  implicit  or  structured  implicit 
methodologies  commonly  used  to  determine  if  an  adverse  drug  event  occurred.  Implicit 
evaluation  suffers  from  a  lack  of  reliability  (Allison  et  al.,  2000;  Ashton  et  al.,  1999; 
Brennan,  2000).  Thus,  practice  pattern  variation  may  play  a  larger  role  in  the  assessment 
of  each  clinical  scenario.  Therefore,  while  the  current  study  and  MacKinnon's  (1999) 
panel  may  not  have  perfect  agreement,  once  the  definitions  are  applied  to  the  data,  a 
consistent  measure  is  afforded. 
The  Panel  and  Preventability  Framework 

Panel  members  were  instructed  to  use  Hepler  and  Strand's  (1990)  framework  to 
determining  whether  events  were  preventable  or  not.  An  adverse  outcome  attributable  to 
drug  therapy  was  deemed  preventable  if;  (1)  the  adverse  outcome  was  preceded  by  a 
recognizable  drug  therapy  problem,  (2)  the  adverse  outcome  of  the  drug  therapy  problem 
had  been  vezsondkAy  foreseeable,  (3)  the  cause  of  the  adverse  outcome  could  have  been 
identifiable  with  reasonable  probability,  and  (4)  the  cause  of  the  adverse  outcome  could 
have  been  reasonably  controllable  within  the  context  and  objectives  of  therapy. 

Written  responses  from  panel  members  demonstrated  their  use  of  this  framework 
in  their  assessment  of  each  definition.  Several  selected  statements  made  by  panel 
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members  (both  for  and  against  the  event's  preventability)  and  their  context  are  detailed 

below  for  each  component  of  the  preventability  framework. 

•  "Inappropriate  treatment  for  condition."  Referring  to  use  of  long-acting 
benzodiazepines  in  patients  with  a  history  of  depression.  (Recognizable) 

•  "Unusual  occurrence  with  short  term  antibiotics.  Usually  occurs  later." 
Referring  to  the  use  of  antibiotics  with  warfarin.  (Recognizable) 

•  "Well  accepted  use  of  agents  these  days."  Referring  to  the  use  of  aspirin  and 
beta-blockers  to  prevent  a  secondary  myocardial  infarction.  (Recognizable) 

•  "Would  not  use  if  prior  history  of  GI  bleeding."  Referring  to  the  use  of  oral 
corticosteroids  in  a  patient  with  a  history  of  GI  problems.  (Recognizable) 

•  "Beta-blockers  mask  hypoglycemic  levels."  Referring  to  use  of  beta-blockers 
in  diabetic  patients.  (Recognizable) 

•  "Recognizable  as  long  as  we  know  exactly  what  they  are  taking.  Often  we 
don't."  Referring  to  the  use  of  NSAIDS  and  their  availability  over-the- 
counter.  (Recognizable) 

•  "HbAjC  is  recommended  every  3  months  for  IDDM  and  every  6  months  for 
NDDM.  But  this  will  not  predict  acute  changes  (hyperglycemia);  for  this, 
daily  glucose  fingersticks  are  better."  (Foreseeable) 

•  "HbAiC  should  be  substituted  with  daily  blood  sugars."  (Foreseeable) 

•  "Not  necessarily  related.  Other  factors  may  be  at  work."  When  discussing 
falls  and  tricyclic  antidepressants.  (Identifiable) 

•  "Not  a  strong  correlation."  When  discussing  falls  and  the  use  of  nitrates. 
(Identifiable) 

•  "Digoxin  with  therapeutic  levels  can  cause  bradycardia."  Referring  to  use  of 
digoxin  in  a  CHF  patient  with  heart  block  or  advanced  heart  block. 
(Identifiable) 

•  In  referring  to  the  use  of  NSAIDs,  one  physician  stated  that  "patients  often 
take  these  on  their  own  (OTC)."  (Controllable) 

•  "NSAID  usage  can  cause  GI  problems  even  with  cytoprotective  agents." 
(Controllable) 
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•  "Patients  fall  and  break  their  hips  without  being  caused  by  tricyclic 
antidepressants."  (Controllable) 

•  "COX-2  agents  are  better  alternative  to  NSAIDs  to  prevent  GI  problems." 
(Controllable) 

•  "Avoid  tricyclics  in  the  elderly."  (Controllable) 

Operationalization 
The  gold  standard  in  the  identification  of  medical  codes  in  an  administrative 
database  is  the  use  of  experienced  medical  record  coders  combined  with  a  physician 
(Steinberg  et  al.,  1990).  Our  study  used  that  methodology  to  identify  all  possible  codes 
for  explicit  definitions  of  preventable  drug-related  morbidity. 

Operationalization  of  the  medical  care  components  for  each  definition  involved 
the  use  of  experts  in  the  application  of  ICD-9  and  CPT  codes.  Two  medical  record 
coders  were  instructed  to  identify  all  possible  codes  for  the  pattern  of  care  and  outcome 
for  each  consensus-approved  definition.  An  intermediate  list  of  codes  was  generated 
using  all  codes  identified  by  at  least  one  coder. 

The  intermediate  list  of  codes  was  given  to  a  physician  for  review  with 
instructions  to  determine  the  clinical  relevance  of  the  code  to  the  definition.  The 
physician  reviewer  eliminated  nearly  20%  of  the  codes  identified  by  the  medical  record 
coders.  The  number  of  codes  eliminated  through  the  physician  review  suggests  that 
asking  coders  to  think  clinically  exceeded  their  capabilities.  The  use  of  a  physician  to 
review  codes  was  critical  to  the  reliability  of  selecting  appropriate  codes. 

Measurement 
Prevalence  of  Preventable  Drug-Related  Morbidity 

Out  of  1 1,71 1  patients  eligible  for  the  study,  966  (8.2%)  had  at  least  one  instance 
of  preventable  drug-related  morbidity.  This  number  is  higher  than  MacKinnon's  (1999) 
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prevalence  of  2.9%.  The  first  thing  to  consider  is  whether  the  length  of  the  data 

collection  period  was  the  cause  for  the  difference.  MacKinnon  (1999)  collected  data  over 

a  12  months  period.  The  current  study  allowed  for  up  to  18  months  of  data  collection. 

The  average  patient  in  the  current  study  was  enrolled  in  the  managed  care  plan  for  483 

days.  This  is  1 .32  times  longer  than  MacKinnon's  (1999)  study  period.  The  current 

study  yielded  a  PDRM  prevalence  rate  of  6.2%  per-member-year,  which  is  still  more  than 

double  MacKinnon's  (1999)  rate. 

A  second  possible  explanation  for  the  higher  prevalence  is  the  process  used  to 
identify  medical  claims  codes.  In  MacKinnon's  (1999)  study,  the  principal  investigator 
(NJM)  chose  medical  claims  codes  with  no  outside  expertise.  The  current  study  used 
experienced  medical  record  coders  and  a  physician  to  identify  all  possible  codes.  Thus,  it 
is  likely  that  a  more  complete  list  of  ICD-9  and  CPT  codes  was  available. 

A  third  possible  explanation  is  a  difference  in  study  populations.  MacKinnon 
(1999)  does  not  provide  demographic  data  on  his  population;  thus  comparison  is  not 
possible.  Both  studies,  however,  focus  on  the  elderly. 
Comparison  of  PDRM  Detection  With  MacKinnon's  Study 

Of  the  966  patients  with  preventable  drug-related  morbidity,  70.9%  had  only  one 
instance  and  29. 1%  had  more  then  one  instance  of  PDRM.  These  numbers  are  similar  to 
MacKinnon's  (1999)  study  where  62.9%  had  only  one  instance  of  preventable  drug- 
related  morbidity  and  37.1%  had  more  than  one.  Thus,  the  study  populations 
demonstrated  a  similar  distribution  of  events.  This  may  suggest  that  a  more  reliable 
coding  process  increased  the  identification  of  PDRM  events. 

The  detection  of  specific  preventable  drug-related  morbidity  in  our  study 
population  and  MacKinnon's  (1999)  population  illustrates  both  important  similarities  and 
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differences.  Appendices  P  and  Q  list  the  most  common  PDRM  events  detected  in  each 

population.  Four  of  the  most  common  events  were  similar  in  both  studies.  Two  of  the 

four  definitions  common  to  both  study  populations  had  congestive  heart  failure  (CHF)  as 

their  adverse  outcome  of  interest  (definition  19-CHF/Hx-NSAID,  and  definition  45- 

CHF/Hx-ACEI).  Thus,  both  study  populations  exhibit  problems  with  the  patterns  of  care 

patients  with  CHF  receive.  Congestive  heart  failure  is  the  most  common  discharge 

diagnosis  by  diagnosis-related  groups  (DRG)  in  the  United  States,  accounting  for  1 

million  hospitalizations  and  200,000  deaths  per  year  (Costantini  et  al.,  2001).  Given  the 

prevalence  of  CHF  and  the  poor  treatment  patterns,  detection  of  PDRM  involving  these 

patients  adds  validity  to  our  measure  (McDermott  et  al.,  1998;  Reis  et  al.,  1997;  Stafford 

etal.,  1997). 

The  event  most  evenly  detected  PDRM  event  in  both  study  populations  had 
hypothyroidism  as  the  adverse  outcome  (definition  22,  Hypoth/Thy-T4TSH).  In  both 
study  populations,  the  event  accounted  for  7.6%  of  total  PDRM  events  detected.  The 
medical  director  of  the  managed  care  organization  initially  questioned  this  result. 
However,  the  similarity  of  the  results  between  patient  populations  added  to  the  validity  of 
the  result. 

A  patient  experiencing  a  secondary  myocardial  infarction  (MI)  without  receiving 
either  aspirin  or  a  beta-blocker  was  also  common  to  both  study  populations.  This  is  a 
widely  accepted  pattern  of  care  and  should  prompt  practitioners  to  review  all  patients 
with  a  previous  MI  to  evaluate  the  pattern  of  care.  However,  aspirin  use  is  difficult  to 
detect  in  an  administrative  database  due  to  its  over-the-counter  (OTC)  status.  Thus,  both 
studies  may  have  patients  with  an  appropriate  pattern  of  care  based  on  the  consensus- 
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approved  definition.  Patients  may  be  receiving  aspirin,  yet  still  have  a  secondary 

myocardial  infarction,  indicating  that  beta-blockers  may  be  of  more  importance  in 

preventing  secondary  myocardial  infarctions. 

The  current  Geriatric  Medicine  Expert  panel  rejected  four  of  the  top  12  definitions 
in  MacKinnon's  (1999)  population.  If  these  specific  scenarios  had  been  measured  in  the 
current  study,  the  actual  difference  in  the  detection  of  events  may  have  been  greater. 
Definitions  With  No  Events  Screened  Positive 

The  study  population  did  not  exhibit  any  instances  of  preventable  drug-related 
morbidity  for  20  (40.8%)  of  the  49  consensus-approved  definitions.  This  number  is 
similar  to  MacKinnon's  (1999)  study  where  23  out  of  52  (44.2%)  definitions  detected  no 
instances  of  preventable  drug-related  morbidity. 

Four  consensus-approved  definitions  with  no  instances  of  PDRM  detected  in  the 
study  population  identified  specific  drug  toxicities  (theophylline,  anticonvulsants, 
lithium,  and  aminoglycosides).  The  lack  of  detection  may  be  in  part  a  better  job  of 
monitoring  for  the  effects  of  these  medications  and  preventing  drug-related  morbidity. 
The  lack  of  events  may  also  be  due  to  the  difficulty  of  detecting  these  outcomes  in  an 
administrative  database.  Signs  and  symptoms  associated  with  drug  toxicity  often  go 
unrecognized  or  are  not  coded  as  such.    There  are  "e-codes"  which  identify  adverse 
events  related  to  the  use  of  a  medication,  but  rarely  are  these  codes  used  (Banks  et  al., 
1999;Beroetal.,  1991). 

Major  or  minor  hemorrhagic  events  due  to  the  misuse  of  heparin  or  warfarin  were 
also  rare  in  the  current  population.  The  use  of  these  agents  has  raised  serious  safety 
concerns  in  the  literature  for  years.  The  evidence  from  this  study  may  lead  us  to 
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conclude  that  the  health  care  system  has  reduced  the  prevalence  of  these  specific  drug- 
related  morbidity  events. 

Renal  failure  was  not  a  common  PDRM  event  detected  in  this  patient  population. 
Renal  dosing  of  medications  and  the  renal  monitoring  of  elderly  patients  is  strongly 
recommended  (DiPiro  et  al.,  1999).  Once  again,  the  results  suggest  that  these  patterns  of 
care  have  been  improved  in  the  current  medication  use  system. 

Pattern  of  Care  and  Outcome  Assessment 

Previous  studies  have  looked  at  process  indicator  identification  using  computer 
databases  (Classen  et  al.,  1999;  Raschke  et  al.,  1998).  However,  these  studies  only  look 
for  a  poor  process  without  identifying  the  corresponding  adverse  outcome.  The  strength 
of  the  current  study  is  the  linking  of  the  pattern  of  care  with  the  corresponding  outcome. 
Each  pattern  of  care  and  corresponding  outcome  was  identified  independently  in  the 
administrative  database.  A  date  of  occurrence  was  attached  to  each  component.  Patients 
with  both  components  present  (thus,  PDRM)  had  the  Number  of  Days  from  Pattern  of 
care  to  Outcome  (NDPO)  recorded.  Patients  with  the  pattern  of  care  present  but  no 
corresponding  outcome  also  had  the  number  of  days  between  its  presence  and  the  end  of 
the  study  period  (September  30,  1999)  calculated. 

Preventing  adverse  outcomes  requires  that  practitioners  have  sufficient  time 
between  the  presence  of  the  pattern  of  care  and  the  outcome  to  avert  the  adverse  event. 
Using  the  framework  for  preventability  established  earlier,  a  practitioner  must  recognize 
the  drug-related  problem  and  foresee  the  potential  for  a  negative  outcome.  Additionally, 
the  practitioner  must  have  adequate  time  to  identify  and  control  the  cause  within  the 
normal  course  of  care.  Thus,  the  time  between  the  pattern  of  care  and  outcome  is  a  useful 
measure.  However,  no  attempt  was  made  to  make  a  clinical  judgment  about  how  long 
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the  length  of  time  between  the  pattern  of  care  and  outcome  could,  or  should,  be  before  it 

should  not  be  counted  as  a  PDRM  event. 

There  was  a  large  variability  in  the  amount  of  time  for  a  pattern  of  care  to  result  in 
an  adverse  outcome.  An  adjustment  in  the  number  of  patients  with  only  the  pattern  of 
care  was  made  to  account  for  patterns  of  care  that  did  not  have  sufficient  time  to  result  in 
the  adverse  outcome  (see  Chapter  3). 
Process-To-Outcome  Value 

The  Process-To-Outcome  Value  (PTO V)  reflects  the  proportion  of  patients  with  a 
pattern  of  care  (after  adjustment)  that  also  had  the  corresponding  outcome.  Definitions 
with  a  high  Process-To-Outcome  Value  demonstrate  a  high  association  between  the 
presence  of  the  pattern  of  care  and  the  adverse  outcome.  Each  definition  with  a  high 
PTOV  has  the  potential  to  serve  as  an  indicator  in  the  medication  use  process. 

Appendix  R  lists  the  Process-To-Outcome  Values  for  each  definition  where  at 
least  10  instances  of  PDRM  were  detected.  Six  of  the  consensus-approved  definitions  (1- 
Dep/Hx-Benzo,  7-Dep/Hx-Symp,  25-COPD/Hx-BB,  32-GI/Hx-OCort,  41-Dep/Hx-BB, 
48-Asth/Hx-Bdilat-ICort)  had  a  Process-To-Outcome  Value  of  1 .00  (refer  to  Appendix  D 
for  numbers  and  appendices  E  and  F  for  mnemonics).  This  means  that  all  patterns  of  care 
(after  adjustment)  eventually  resulted  in  the  adverse  outcome. 

Three  of  the  six  definitions  with  a  PTOV  of  1 .00  had  a  pattern  of  care  that 
included  a  previous  history  of  depression.  Thus,  patients  with  a  diagnosis  of  depression 
warrant  increased  attention  by  health  care  practitioners.  Indicators  that  identify  patients 
with  depression  and  the  corresponding  drug-related  problems  should  be  developed  based 
on  the  results  of  this  study. 


81 
The  most  common  preventable  drug-related  morbidity  event  with  a  PTOV  of  1 .00 

involved  patients  with  a  history  of  asthma  who  are  not  on  an  inhaled  corticosteroid,  but 

are  receiving  a  bronchodilator  (89  events  detected  in  the  patient  population).  Treatment 

guidelines  for  asthma  patients  clearly  call  for  inhaled  corticosteroids  as  preventative 

therapy  (DiPiro  et  al.,  1999).  This  study  demonstrates  that  patients  who  do  not  receive 

inhaled  corticosteroids  will  inevitably  have  an  adverse  outcome  that  could  have  been 

prevented.  The  use  of  inhaled  corticosteroids  is  recommended  as  an  indicator  of  quality 

in  the  medication  use  system.  This  excessive  use  of  bronchodilators  may  also  serve  as  an 

indicator  of  poorly  controlled  asthma  patients. 

The  definition  including  asthma  as  the  adverse  outcome  illustrates  the  issue  of 
evidence  versus  practice  patterns.  While  the  definition  had  a  PTOV  of  1 .00,  it  only 
received  minimal  agreement  in  the  Delphi  panel  (5  out  of  7  panel  members).  Thus,  even 
in  a  process  where  evidence  was  used  to  develop  explicit  definitions  and  panel  members 
were  given  instructions  on  a  framework  for  the  process,  subjective  practice  pattern 
variation  is  still  evident.  This  result  makes  public  the  decision  process  physicians  face 
when  prescribing  medications  for  their  patients. 

There  were  other  consensus-approved  definitions  where  the  Process-To-Outcome- 
Value  was  relatively  high,  although  not  1 .00.  These  patterns  of  care  may  also  prove 
useful  as  indicators  of  poor  medication  use.  Definition  33  (CHF/Hx-Dig)  resulted  in  184 
adverse  outcomes  from  193  patterns  of  care  (PTOV  =  0.95).  This  pattern  of  care  would 
also  be  a  good  indicator  of  quality  in  the  medication  use  system. 

Patients  with  a  history  of  congestive  heart  failure  who  were  not  receiving  an  ACE 
inhibitor  (definition  number  45)  resulted  in  the  most  instances  of  preventable  drug-related 
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morbidity  in  our  patient  population  (270).  This  pattern  of  care  also  had  a  relatively  high 

PTOV  (0.75).    The  use  of  an  ACE  inhibitor  is  well  accepted  for  most  patients  with  CHF. 

However,  one  study  demonstrated  that  only  18%  of  CHF  patients  discharged  from  the 

hospital  received  appropriate  ACE  inhibitor  therapy  (McDermott  et  al.,  1998).  Other 

studies  indicate  that  ACI  inhibitors  are  underutilized  in  the  treatment  of  CHF  (Reis  et  al., 

1997;  Stafford  etal.,  1997). 

Combined  with  definition  33  (CHF/Hx-Dig),  patients  with  congestive  heart 
failure  should  receive  additional  attention  regarding  their  medication  use  in  order  to 
prevent  adverse  outcomes  from  occurring.  Added  attention  could  result  in  significant 
reductions  in  hospitalizations,  since  congestive  heart  failure  is  the  most  common  DRG 
discharge  diagnosis  and  adverse  events  related  to  ACE  inhibitors  are  a  common  reason 
for  admission  (Costantini  et  al.,  2001 ;  Darchy  et  al.,  1999).  Programs  targeted  at  CHF 
patients  could  reduce  the  per-patient  cost  of  care  and  improve  patient  outcomes  (Fonarow 
etal.,  1997). 
Definitions  With  a  Low  PTOV 

Several  consensus-approved  definitions  had  a  number  of  patients  with  the  pattern 
of  care  present  but  not  the  corresponding  outcome,  even  after  adjustment  for  NDPO. 
Thus,  these  definitions  had  a  low  PTOV.  The  five  definitions  with  the  most  patterns  of 
care  and  their  PTOV  are  listed  in  Table  5.1 . 

Each  of  these  medication  use  issues  represents  a  common  treatment  regimen.  The 
presence  of  a  low  PTOV  suggests  that  the  pattern  of  care  is  not  strongly  associated  with 
the  negative  outcome.  However,  they  still  may  be  of  value  as  an  indicator  if  they  are  of 
high  cost,  high  volume,  or  high  risk. 
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One  definition  that  may  serve  as  an  indicator  due  to  its  relatively  high  frequency 

is  number  39  (GI/NSAID).  There  were  129  patients  who  were  on  2  or  more  NSAIDS  for 
more  than  2  weeks  with  the  corresponding  outcome  of  gastritis  and/or  upper  GI  bleed 
and/or  GI  perforation  and/or  GI  ulcer  and  anemia.  While  the  association  between  the 
pattern  of  care  and  the  adverse  outcome  was  not  strong  (PTOV  =  0.12),  the  sheer  number 
of  patients  with  this  pattern  of  care  warrants  attention.  This  is  supported  by  literature  that 
implicates  non-steroidal  anti-inflammatory  drugs  (NSAIDs)  as  a  leading  cause  of  drug- 
related  hospital  admissions  (Cunningham  et  al.,  1997). 


Table  5.1 :  Preventable  Drug-Related  Morbidity  Definitions  With  a  Low  PTOV 

Definition11 

Mnemonic" 

Number  of  patterns 
of  care-only 

Number  of  PDRMC 
events  detected 

PTOVd 

39 

GI/NSAID 

929 

129 

0.12 

44 

FB/SedHyp 

655 

42 

0.06 

20 

HypoK/Kwd-NoK- 
Elec 

365 

44 

0.11 

28 

HyprK/ACEI- 
EleCBC 

360 

14 

0.04 

19 

CHF/Hx-NSAID 

139 

80 

.37 

"Definition  number  matches  Appendix  D 
bMnemonics  in  appendices  E  and  F 
CPDRM:  preventable  drug-related  morbidity 
dPTOV:  process-To-Outcome  Value 

Preventable  Drug-Related  Morbidity  as  an  Extension  of  Drug  Use  Evaluation 

Drug  use  evaluation  (DUE)  has  been  used  in  health  care  to  determine  the 

appropriateness  of  medication  use.  However,  most  DUE  programs  focus  on  prescribing, 

rather  than  viewing  the  entire  system  of  medication  use.  Beers  et  al.  (1991)  attempted  to 

develop  explicit  criteria  to  evaluate  inappropriate  medication  use  in  nursing  home 

patients.  Beers  (1997)  later  update  the  criteria  to  look  at  all  elderly  patients,  rather  than 

just  the  frail  in  nursing  homes.  Each  set  of  criteria  identified  medications  that  should  be 
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avoided  in  elderly  patients  with  certain  conditions,  and  doses,  frequencies,  or  durations  of 

therapy  that  should  not  be  exceeded  in  the  elderly.  In  both  cases,  the  authors  provide 

explicit  criteria  that  evaluate  only  prescribing,  rather  than  the  use  of  medications. 

Additionally,  the  lack  of  clinical  information  made  the  assessment  of  appropriateness 

questionable  at  best.  In  most  cases,  one  cannot  state  unequivocally  that  a  drug  should  not 

be  used  without  consideration  of  other  factors. 

Hanlon  et  al.  (1992)  also  developed  explicit  criteria  to  evaluate  the  use  of 
medications,  called  the  Medication  Appropriateness  Index  (MAI).  The  index  is  based  on 
10  factors  to  assess  the  appropriateness  of  medication  use  (Hanlon  et  al.,  1992).  Their 
work  was  an  important  advancement  over  previous  studies  because  they  attempted  to 
assess  factors  other  than  prescribing  the  right  drug.  They  gathered  limited  clinical 
information  to  assist  in  their  determination  of  appropriate  use.  However,  they  did  not 
address  ongoing  monitoring  or  the  outcomes  of  the  medication  use. 

The  current  study  advances  previous  methods  for  drug  use  evaluation  in  that  it 
assesses  medication  use,  not  just  prescribing,  and  the  outcome  of  care.  Beers  (1997) 
makes  the  point  that  clinical  information  should  be  used  to  judge  appropriateness.  Our 
study  does  this  by  looking  for  laboratory  requirements  or  a  past  medical  history  where 
indicated.  The  use  of  laboratory  monitoring  also  advances  the  Medication 
Appropriateness  Index  from  Hanlon  et  al.  (1992). 

The  linking  of  an  outcome  with  the  pattern  of  care  is  the  most  significant 
advancement  in  drug  use  evaluation  offered  by  this  study.  Buetow  et  al.  (1997)  suggest 
that  inappropriate  prescribing  is  related  to  the  outcome  of  medication  use  and  not  the 
process.  By  linking  the  outcome  and  the  pattern  of  care,  true  inappropriate  drug  use  can 
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be  assessed.  The  study  also  assessed  the  association  of  poor  patterns  of  care  and  the 

appearance  of  adverse  outcomes.  The  results  of  the  study  should  focus  attention  on  more 

clinically  relevant  issues. 

Validity  of  a  Measure  of  Preventable  Drug-Related  Morbidity 

Validity  of  the  measure  of  preventable  drug-related  morbidity  is  supported  by 
much  of  the  data  previously  presented.  However,  additional  evidence  will  be  discussed 
using  Messick's  validity  framework  (Messick,  1989).  Messick  treats  validity  as  a  unitary 
concept  with  both  an  evidentiary  and  consequential  basis  for  the  interpretation  and  use  of 
a  measure  (Messick,  1989). 
Evidentiary  Basis  for  Interpretation 

The  first  quadrant  in  Messick's  framework  addresses  the  evidence  gathered  to 
support  the  interpretation  of  our  measure.  The  evidence  needed  for  interpretation 
addresses  construct  validity  with  support  from  content  or  face  validity. 
Face  and  content  validity 

The  expert  panel  defining  preventable  drug-related  morbidity  supports  the 
evidential  basis  for  interpretation.  Panel  members  came  to  agreement  on  49  explicit 
definitions  of  PDRM.  Thus,  the  panel  agreed  that  the  definitions  were  relevant  to  the 
issue  of  preventable  drug-related  morbidity  and  were  useful  in  detecting  instances  of 
PDRM.  The  current  panel's  results  add  to  the  evidence  from  MacKinnon's  (1999) 
content  panel. 

Validity  is  also  supported  by  the  panel  member's  use  of  literature  in  their  decision 
process.  Evidence-based  medicine  uses  the  literature  systematically  to  make  clinical 
decisions  (Mootonen  et  al.,  2001;  Rosenberg  &  Donald,  1995).  The  use  of  evidence  in 
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the  process  supports  validity  of  the  definitions  as  being  relevant  to  preventable  drug- 
related  morbidity. 

The  panel  also  recommended  additional  definitions  that  were  accepted,  adding  to 
the  representativeness  of  the  test.  By  recommending  additional  definitions,  the  panel  was 
reasoning  within  the  framework  of  the  study  and  using  their  expertise  to  identify 
additional  instances  of  preventable  drug-related  morbidity. 
Risk  factors 

Association  with  known  risk  factors  also  supports  the  validity  of  the  measure  of 
preventable  drug-related  morbidity.  The  current  study  used  MacKinnon's  (1999)  risk 
factors  as  confirmatory  hypotheses  to  support  the  use  of  preventable  drug-related 
morbidity  definitions.  Using  a  simple  logistic  regression  model,  all  factors  were 
independently  associated  with  the  presence  of  preventable  drug-related  morbidity.  This 
mirrored  the  results  of  the  MacKinnon  (1999)  study. 

The  intent  of  the  study  was  not  to  determine  the  causative  direction  for  these 
confirmatory  hypotheses.  Rather,  it  was  important  to  show  that  patients  with  4  or  more 
prescribers,  6  or  more  prescriptions,  4  or  more  diagnoses,  etc.,  were  more  likely  to  have 
an  instance  of  preventable  drug-related  morbidity  detected.  The  strong  association  of  the 
presence  of  a  hypothesized  risk  factor  and  preventable  drug-related  morbidity  adds  to  our 
ability  to  interpret  the  measure. 
Resource  use 

The  patient  population  was  studied  to  determine  if  a  relationship  between  the 
amount  of  resources  used  and  the  presence  of  preventable  drug-related  morbidity  existed. 
As  the  case  with  confirmatory  risk  factors,  the  exploration  of  costs  adds  evidence  for  the 
interpretation  of  a  measure  of  PDRM.  Theoretically,  patients  with  preventable  drug- 
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related  morbidity  should  use  more  resources  than  patients  who  have  not  experienced  an 

adverse  event. 

Patients  with  preventable  drug-related  morbidity  experienced  more 
hospitalizations  than  patients  without  PDRM.  There  are  a  number  of  studies  that 
demonstrate  that  a  significant  number  of  patients  are  admitted  to  the  hospital  as  a  result 
of  drug-related  events  (Bergman  &  Wiholm,  1981;  Bero  et  al.,  1991;  Bigby  et  al.,  1987; 
Cunningham  et  al.,  1997;  Darchy  et  al.,  1999;  Hallas  et  al.,  1992).  While  the  intent  was 
not  to  determine  causative  direction,  evidence  supports  the  proper  association. 

Patients  with  preventable  drug-related  morbidity  tended  to  use  other  health  care 
resources  (emergency  room  visits,  physician  office  visits,  and  medications)  more  often 
than  those  with  no  evidence  of  PDRM.  This  has  been  postulated  and  studied  by  other 
authors  (Bates  et  al.,  1997;  Bootman  et  al.,  1997;  Dennehy  et  al.,  1996;  Johnson  & 
Bootman,  1995;  Tafreshi  et  al.,  1999).  Again,  the  direction  of  causation  was  not 
determined.  However,  showing  the  association  between  patients  with  PDRM  and  the  use 
of  additional  resources  adds  to  the  evidence  for  validity. 

Overall,  the  cost  of  caring  for  patients  with  preventable  drug-related  morbidity 
was  higher  than  those  without  PDRM.  This  was  true  from  the  payer's  perspective,  the 
patient's  viewpoint,  and  for  total  direct  health  care  costs.  The  direction  of  causation  once 
again  was  not  determined.  Other  studies  have  shown  that  patients  with  PDRM  have  a 
higher  total  cost  than  those  without  (Bates  et  al.,  1997;  Classen  et  al.,  1999;  Thomas  et 
al.,  1999). 
Evidential  Basis  for  Use 

The  discussion  regarding  the  evidence  for  interpretation  of  the  measure  of  PDRM 
may  also  be  applied  to  the  use  of  the  measurement.  Thus,  the  expert  panel  and 
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measurement  in  the  patient  population  support  validity  for  use  as  a  measure  of  PDRM.  In 

addition  to  this,  the  relevance  and  utility  of  a  measure  are  explored  through  a  root  cause 

analysis. 

Root  cause  analysis 

The  results  from  the  root  cause  analysis  help  to  explore  the  relevance  and  utility 
of  the  measure  of  preventable  drug-related  morbidity  as  it  relates  to  quality  of  the 
medication  use  system.  The  root  cause  analysis  is  an  important  tool  in  quality  assessment 
and  improvement  initiatives  (Bates,  1996;  Fernandes  et  al.,  1997;  JCAHO,  1996; 
JCAHO,  1998;  Shinn,  2000).  A  root  cause  analysis  is  often  used  in  a  systematic  process 
to  uncover  quality  problems  in  medication  use. 

Quality  assessment  and  improvement  is  an  important  function  in  health  care 
(JCAHO,  1996;  JCAHO,  1998).  The  acronym  FOCUS-PDCA  was  developed  by  the 
Columbia/HCA  Corporation  in  Nashville,  Tennessee  to  provide  a  systematic  framework 
for  quality  assessment  and  improvement  activities  (Gerard  &  Arnold,  1996;  Redick, 
1999).  The  use  of  the  measure  of  preventable  drug- related  morbidity  in  a  systematic 
framework  such  as  FOCUS-PDCA  was  accomplished.  The  FOCUS-PDCA  process  is 
reviewed  along  with  the  study  components. 

•  Find  a  process  to  improve:  In  the  study,  a  measurement  of  PDRM  provided 
prevalence  estimates  of  patterns  of  care  that  resulted  in  adverse  outcomes. 
The  measure  served  as  an  indicator  that  identified  potential  improvement 
opportunities. 

•  Organize  a  team  that  understands  the  process:  A  panel  of  physicians  and 
pharmacists  involved  in  ambulatory  care  of  elderly  patients  was  convened  to 
address  quality  issues. 

•  Clarify  current  knowledge  of  the  system:  The  root  cause  analysis  panel  did 
not  formally  flow  chart  the  medication  use  process.  However,  there  was 
discussion  on  the  flow  of  patient  care  in  the  current  system. 
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•  Understand  the  causes  of  variation:  The  root  cause  analysis  panel  developed 
cause-and-effect  diagrams  for  the  most  prevalent  PDRM  events  (Appendices 
K,  L,  M,  and  N).  Determining  the  relative  importance  of  each  cause  was 
beyond  the  scope  of  our  study. 

•  Select  the  process  improvement  of  intervention:  Several  potential  system 
improvements  were  suggested  at  the  root  cause  analysis  meeting.  However, 
this  and  all  remaining  steps  in  the  FOCUS-PDCA  process  were  beyond  the 
scope  of  our  study. 

•  Plan  the  improvement. 

•  Do  implement  the  improvement. 

•  Check  the  improvement  by  gathering  and  analyzing  data. 

•  Act  on  the  data  to  hold  the  gain  or  continue  improving. 

While  the  process  employed  in  the  study  did  not  allow  time  for  the  root  cause 
analysis  panel  to  come  to  full  consensus,  there  was  ample  evidence  to  demonstrate  that 
the  measure  of  preventable  drug-related  morbidity  was  useful  in  identifying  quality 
problems  in  the  medication  use  system.  This  demonstrates  their  relevance  and  utility  to 
identify  performance  improvement  opportunities. 

Representatives  from  the  managed  care  partner  are  exploring  possible 
interventions  to  reduce  the  problem  of  PDRM  in  their  patient  population  using  the 
measure  and  results  of  the  root  cause  analysis.  This  further  supports  the  utility  of  a 
measure  of  preventable  drug-related  morbidity  to  improve  medication  use. 
Consequential  Basis  for  Interpretation 

While  a  direct  measure  of  the  consequential  basis  for  interpretation  of  our  results 
was  not  planned  as  part  of  the  study,  some  important  points  can  be  made.  The 
consequential  basis  for  interpretation  of  the  measure  of  PDRM  assesses  the  value 
associated  with  the  label.  In  terms  of  preventable  drug-related  morbidity,  the  key  word 
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may  be  preventable.  By  implying  that  an  outcome  was  preventable,  a  negative  value  is 

attached  if  the  event  occurs.  The  root  cause  analysis  panel  discussed  quality  issues 

involved  with  the  definitions  and  what  the  measure  would  mean  to  the  MCO.  In 

particular,  poor  quality  and  a  system  that  is  not  safe  could  be  implied.  Also  implied  is 

that  an  error  may  have  occurred  that  the  physician  is  responsible  for  due  to  his  decision 

making  (Bates  et  al.,  1995a).  These  would  certainly  have  implications  for  the 

consequences  of  interpreting  a  measure  of  PDRM. 

Consequential  Basis  for  Use 

The  consequential  basis  for  use  addresses  the  social  consequences  of  the  measure. 
Once  again,  no  direct  measure  was  planned  to  assess  this  dimension  of  validity. 
However,  the  physician  who  was  disqualified  from  the  Delphi  panel  obviously  was 
concerned  about  social  consequences  based  on  the  summative  use  of  the  measure  (the 
measure  is  intended  for  formative  purposes,  i.e.,  quality  improvement,  and  not  for 
judging  individual  cases  or  practitioners).  This  physician's  concern  was  centered  on  the 
use  of  the  measure  to  judge  individual  practitioner's  care  patterns.  Recall  that  he  said,  "I 
don't  care  what  you  say  these  definitions  will  be  used  for,  I  know  that  [MCO  name]  will 
use  them  as  practice  standards  and  tell  me  I  am  not  doing  things  right."  He  was 
concerned  about  the  consequences  of  the  summative  use  of  a  PDRM  measure  on  his 
practice.  This  use  should  be  acknowledged  as  one  possible  use  if  the  measure  is  released 
for  broader  application. 

Potential  social  consequences  could  also  be  seen  at  the  organizational  level  if  the 
measures  are  used  in  a  summative,  rather  than  formative,  manner.  The  use  of  a  measure 
of  preventable  drug-related  morbidity  in  report  cards  to  assess  the  quality  of  care 
provided  by  an  organization  could  have  consequences  for  social  policy  if  they  were  used 
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for  summative  purposes.  The  consequences  would  stem  from  public  and  payer  reaction 

to  the  information.  Organizations  with  a  lower  measure  would  benefit  from  the  release  of 
such  information.  However,  those  organizations  with  a  poor  measure  would  risk  losing 
their  business.  The  measure  was  not  designed  for  this  purpose,  however  it  must  be 
recognized  as  a  potential  consequence. 

Conclusion 

This  study  began  with  four  research  objectives.  The  first  was  to  develop  and 
validate  definitions  of  PDRM.  This  research  objective  was  accomplished  through  a 
Delphi  panel  that  replicated  MacKinnon's  (1999)  work.  The  Delphi  panel  approved  49 
explicit  definitions  of  preventable  drug-related  morbidity.  These  definitions  were  judged 
to  be  both  relevant  and  representative  of  the  domain  of  preventable  drug-related 
morbidity. 

The  second  research  objective  was  to  identify  issues  in  operationalizing  the 
definitions  of  PDRM  to  be  used  in  a  computer  database.  Two  medical  record  coders  and 
a  physician  were  used  to  generate  ICD-9  and  CPT  codes  relevant  to  each  PDRM 
definition.  The  physician  was  a  critical  component  in  accomplishing  this  objective  due  to 
variation  in  selecting  codes.  Nearly  20%  of  the  codes  selected  by  medical  record  coders 
were  eliminated  through  a  clinical  review. 

The  third  objective  was  to  use  the  explicit  definitions  to  measure  PDRM  in  the 
study  population.  A  total  of  966  (8.2%)  patients  had  at  least  one  instance  of  preventable 
drug-related  morbidity  identified.  Because  the  patients  were  in  the  study  for  more  then 
one  year,  a  prevalence  rate  of  6.25%  was  calculated.  The  number  of  patients  with  PDRM 
detected  using  explicit  definitions  is  impressive.  The  result  lends  credibility  to  the 
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methodology  used  to  develop  explicit  definitions  for  preventable  drug-related  morbidity 

and  their  application  to  a  managed  care  organization's  database. 

The  measurement  of  PDRM  in  the  population  provided  a  wealth  of  information 
on  the  associations  between  the  pattern  of  care  and  the  corresponding  outcome.  Each 
occurrence  of  a  pattern  of  care  and  outcome  was  identified  independently.  This  allowed 
the  assessment  of  patients  with  either  component  of  the  definition  or  both  components 
(defined  as  PDRM).  The  percentage  of  patients  with  the  pattern  of  care  that  also  had  the 
corresponding  outcome  detected  was  used  as  an  estimate  of  association.  The  study 
identified  specific  instances  of  preventable  drug-related  morbidity  where  the  pattern  of 
care  inevitably  resulted  in  the  corresponding  adverse  outcome. 

The  final  research  objective  sought  to  explore  the  use  the  measure  of  PDRM  in  a 
structured  quality  assessment  and  improvement  process  to  identify  potential  problems  in 
the  medication  use  system.  This  was  accomplished  through  a  panel  that  performed  a  root 
cause  analysis  when  presented  with  the  ten  most  common  definitions.  The  panel 
generated  cause-and-effect  diagrams  that  can  be  used  to  drive  improvement 
opportunities.  The  panel  did  recommend  potential  improvement  programs. 

The  study  is  important  in  the  developing  research  areas  of  explicit  criteria  and 
database  research.  The  methodology  employed  to  identify  both  the  pattern  of  care  and 
the  corresponding  outcome  is  a  significant  improvement  over  previously  published  work. 
The  definitions  identified  a  broader  range  of  patient  data  that  can  be  used  to  assess 
medication  use  beyond  prescribing.  Thus,  these  may  be  used  as  indicators  of  potential 
problems  in  the  medication  use  system.  These  indicators  may  be  used  at  the  health  plan 
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level  in  terms  of  benchmarking  and  quality  improvement.  However,  their  ultimate  use 

may  be  as  targeted  interventions  in  direct  patient  care. 

Potential  Limitations 

Despite  a  literature  review  and  the  use  of  an  expert  panel,  the  49  definitions 
represent  only  a  portion  of  the  PDRM  events  that  occur.  There  is  no  way  of  knowing  the 
proportion  of  the  domain  that  has  been  captured.  However,  two  content  panels  should 
have  identified  the  most  common  events.  Additional  work  and  literature  review  will  be 
necessary  to  add  to  the  definitions.  However,  a  prevalence  rate  of  6.2%  is  a  fairly  sizable 
figure,  suggesting  the  most  common  PDRM  events  were  identified. 

As  with  any  research  on  an  administrative  database,  the  results  are  only  as  good 
as  the  data.  There  are  many  problems  when  relying  on  ICD-9  and  CPT  codes,  such  as 
undercoding,  overcoding,  misspecification  and  missequencing  (Romano  &  Luft,  1 992). 
The  data  given  by  the  managed  care  partner  was  used  as  received.  The  data  were  viewed 
for  cleanliness  by  searching  for  duplicate  visit  information  and  viewing  data  entry  for 
consistency.  As  this  type  of  research  gains  in  popularity,  the  data  quality  will  also 
improve. 

Our  study  population  may  not  be  representative  of  the  elderly  in  America,  thus 
generalizability  is  an  issue.  The  study  also  only  looks  at  the  elderly.  The  question  of 
other  age  groups  (particularly  pediatric  patients)  should  be  explored  in  future  studies. 

There  are  limitations  in  determining  of  the  link  between  the  pattern  of  care  and 
outcome  in  each  patient  with  an  event.  The  correlation  between  the  variables  is  probably 
not  one-to-one.  We  do  not  know  the  sensitivity  of  the  data.  However,  the  results  may 
provide  an  estimate  of  specificity  as  a  lower  bound  estimate.  This  does  provide  fertile 
ground  for  future  studies  to  look  at  these  issues  and  determine  their  predictive  ability. 
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The  study  did  not  attempt  to  determine  a  reasonable  amount  of  time  between  the 

pattern  of  care  and  the  outcome.  This  could  be  a  limitation  to  our  Process-To-Outcome 

Value.  In  health  care,  if  one  waits  long  enough,  the  outcome  will  eventually  occur.  I 

chose  not  to  use  clinical  judgment  to  determine  how  long  was  long  enough  for  each 

definition.  Thus,  there  are  pattern  to  outcome  intervals  in  the  study  that  might  have  been 

too  long  and  do  not  appear  to  make  clinical  sense.  However,  other  cases  that  had  a  large 

number  of  days  between  the  pattern  of  care  and  the  outcome  are  appropriate.  One 

example  is  definition  5  (GI/Hx-NSAID)  where  the  appearance  of  anemia  may  take  a  long 

time  in  many  patients  with  a  slow  GI  bleed. 

It  is  important  to  recognize  that  the  measure  of  preventable  drug-related  morbidity 
based  on  the  consensus-approved  definitions  was  intended  for  use  at  a  population  level  to 
identify  opportunities  to  improve  the  quality  of  the  medication  use  system.  By  itself,  the 
measure  does  not  address  whether  an  organization  is  doing  well  or  poorly.  It  is  meant  to 
gather  information  and  to  be  used  in  a  formative  manner.  Likewise,  the  measure  was  not 
meant  to  evaluate  the  care  patterns  of  individual  practitioners,  especially  not  for  punitive 
purposes.  If  the  definitions  are  used  in  a  summative  manner,  their  acceptance  by  health 
care  organizations  and  practitioners  could  be  limited. 

The  definitions  are  not  meant  to  serve  as  rigid  guidelines  for  care.  For  example, 
the  definition  addressing  the  use  of  an  ACE  inhibitor  in  patients  with  congestive  heart 
failure  does  not  recommend  that  every  patient  receive  the  drug.  However,  it  does  signal 
an  area  to  evaluate  clinically  to  determine  if  the  agent  is  indicated.  Thus,  they  could  be 
used  to  improve  the  care  of  individual  patients  by  targeting  patterns  of  care  for  additional 
attention. 
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The  root  cause  analysis  process  did  tot  prioritizing  the  system  issues  identified  in 

the  session.  They  did  not  selecting  improvement  opportunities  either.  This  may  limit  my 

conclusions  a  regarding  the  use  of  the  measure  of  PDRM.  The  most  important  issues 

would  lead  to  program  implementation  and  impact  assessment. 

Significance  of  Study 

The  primary  significance  of  our  study  is  the  further  development  of  explicit 
definitions  for  preventable  drug-related  morbidity.  By  adding  to  MacKinnon's  (1999) 
study,  more  evidence  is  gathered  in  support  of  the  use  of  explicit  definitions  to  measure 
preventable  drug-related  morbidity.  Using  a  different  expert  panel  strengthens  the  face 
and  content  validity  of  the  definitions.  The  measurement  adds  to  evidence  by  using  a 
different  patient  population. 

The  current  study  expanded  on  previous  work  by  MacKinnon  (1999)  in  the 
operationalization  of  the  definitions.  The  current  study  used  a  rigorous  process  to 
identify  a  broader  range  of  acceptable  codes  that  can  be  replicated.  A  computer  program 
was  designed  to  automatically  search  an  administrative  database  for  preventable  drug- 
related  morbidity.  Thus,  my  measure  is  more  reliable. 

The  study  makes  an  important  contribution  to  the  literature  by  linking  medication 
use  with  other  clinical  components  (such  as  laboratory  monitoring)  and  assessing  its 
association  with  the  outcome.  Most  previous  studies  only  view  the  process  of  care 
without  consideration  of  the  outcome. 

The  study  adds  to  sparse  information  available  on  PDRM  occurring  across  the 
continuum  of  care.  Much  of  the  work  done  on  adverse  drug  events  has  been  in  or  around 
organized  health  care  settings  (such  as  a  hospital).  Ambulatory  care  accounts  for  a 
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significant  portion  of  patient  encounters  and  should  not  be  discounted.  Thus,  the  causes 

of  PDRM  in  an  ambulatory  elderly  population  have  been  explored  more  fully. 

The  study  was  also  beneficial  in  working  with  a  managed  care  organization.  The 
issue  of  preventable  drug-related  morbidity  was  brought  to  light  in  the  organization  and 
important  working  relationships  were  developed.  The  onus  for  action  rests  with  the 
organization  to  use  the  data. 

The  study  adds  to  the  measurement  options  available  for  quality  improvement  in 
the  medication  use  process.  We  were  able  to  demonstrate  that  a  measure  of  PDRM  can 
lead  to  a  root  cause  analysis.  The  design  and  implementation  of  interventions  should 
follow  to  improve  the  quality  of  the  medication  use  system. 

Theoretical  Contribution 

The  theoretical  contributions  are  in  the  areas  of  measurement  and  quality 
improvement.  In  particular,  the  validity  of  explicit  definitions  has  been  explored  and  new 
doors  for  research  have  been  opened. 

The  measurement  of  patterns  of  care  and  corresponding  adverse  outcomes  in  an 
administrative  database  was  an  important  advancement  over  published  work.  This  is 
especially  true  as  it  relates  to  preventable  drug-related  morbidity  in  the  elderly. 

The  validity  of  a  measure  of  preventable  drug-related  morbidity  was  approached 
using  a  comprehensive  framework.  The  use  of  Messick's  framework  for  validity  offers  a 
credible  structure  for  evidence  regarding  the  interpretation  and  use  of  the  measure. 
Additional  studies  will  add  to  the  validation  of  our  measure  of  PDRM. 

The  issue  of  preventability  of  drug-related  events  was  addressed  prospectively 
using  a  well-defined  framework  and  a  panel  of  experts.  This  framework  made  the 
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process  explicit,  rather  then  relying  in  expert  opinion.  This  is  an  important  advancement 

over  implicit  evaluations. 

Finally,  the  idea  of  quality  assessment  and  improvement  was  approached.  The 
measure  of  PDRM  was  used  by  an  expert  panel  to  determine  if  the  data  could  be  used  to 
identify  the  cause  of  quality  problems.  The  resultant  cause-and-effect  diagrams 
demonstrate  their  possible  application  to  improving  the  quality  of  the  medication  use 
system. 

Contribution  to  Health  Care 

Contributions  to  health  care  rest  in  the  improvement  of  patient  safety  and 
reducing  costs.  A  new  method  of  defining  and  measuring  the  problem  of  PDRM  is  put 
forth  as  useful.  This  new  measure  will  help  uncover  a  long  hidden  problem  and  improve 
the  quality  of  the  medication  use  system.  The  result  will  be  a  health  care  system  that  is 
safer  for  patients  and  more  rewarding  for  health  care  practitioners.  The  financial  impact 
on  the  health  care  system  could  be  staggering,  saving  several  billion  dollars  a  year. 

Health  care  in  general  may  also  benefit  through  the  new  measurement  techniques 
used.  This  includes  the  development  and  use  of  explicit  measure  applied  to  an  automated 
database.  Application  of  a  measure  by  several  health  care  organizations  will  make 
benchmarking  possible.  Benchmarking  and  applying  best  practices  will  help  improve  the 
quality  of  the  medication  use  system. 

Health  care  may  also  benefit  by  the  use  of  medical  record  coders  in  the  standard 
procedure  for  operationalizing  the  definitions.  The  ability  for  others  to  replicate  the 
process  adds  to  reliability  in  coding.  The  coding  process  developed  for  this  study  is 
being  replicated  in  England  and  Canada. 
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Contribution  to  the  Profession  of  Pharmacy 

The  contribution  to  the  profession  of  pharmacy  can  be  seen  in  the  call  for 
improved  communication  and  monitoring  systems  for  preventing  drug-related  injuries. 
The  improvement  in  communication  systems  will  open  up  opportunities  for  pharmacist 
involvement  as  a  member  of  the  patient  care  team.  If  pharmacist  involvement  in  the 
patient  care  process  increases,  job  satisfaction  and  standing  will  improve. 

Future  Areas  for  Study 

The  results  of  our  study  open  up  a  wide  array  of  opportunities  for  future  research. 
Chief  among  these  will  be  the  use  of  explicit  measures  applied  to  administrative 
databases.  This  will  be  especially  true  for  ambulatory  patients.  Explicit  measures  should 
be  made  available  for  study  across  several  managed  care  organizations. 

Indicator  development  based  on  the  explicit  definitions  should  also  be  explored. 
The  study  provided  some  initial  information  about  the  association  between  the  pattern  of 
care  and  corresponding  outcome.  Both  the  outcome  and  pattern  of  care  should  be  studied 
as  indicators.  Outcome  indicators  could  signal  populations  of  patients  who  need  disease 
management  interventions.  Pattern  of  care  indicators  could  also  be  studied  on  a 
population  basis  to  determine  the  quality  of  the  medication  use  system.  However, 
patterns  of  care  should  also  be  studied  as  an  indicator  of  potential  problems  for  individual 
patients. 

Each  individual  definition  should  be  studied  to  determine  additional  information 
that  is  needed  to  make  it  more  clinically  relevant.  One  area  that  is  recommended  is  the 
amount  of  time  between  the  pattern  of  care  and  the  outcome  that  a  clinician  could 
reasonably  expect  the  event  to  occur.  This  would  help  to  further  define  the  association 
between  the  pattern  of  care  and  the  outcome,  and  the  PTOV  measure  for  each.  This 
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could  be  done  by  inserting  a  question  into  the  Delphi  process  asking  them  for  a 

reasonable  time  and  a  range  of  time  for  the  outcome  to  occur  given  the  pattern  of  care. 

The  validity  of  the  definitions  should  be  further  tested  with  the  review  of  patient 
charts  as  a  criterion  for  comparison.  Thus,  sensitivity  and  specificity  of  each  definition 
may  be  assessed.  This  would  help  determine  the  need  for  implicit  evaluation  in  further 
studies. 

The  validity  of  the  measures  should  foster  future  research  in  the  area  of  labels 
attached  to  the  measure  and  the  social  implications.  The  issue  of  the  label  attached  to  the 
measure  would  help  with  the  consequential  basis  for  interpretation.  In  this  area,  the  idea 
of  preventability  and  poor  quality  should  be  studied.  The  social  implications  of  the 
measure  address  the  consequential  basis  for  the  use  of  a  measure.  Future  studies  should 
address  organizations  that  choose  to  measure  and  report  preventable  drug-related 
morbidity  and  the  impact  on  their  business.  Inclusion  as  part  of  a  "report  card"  system 
may  open  these  organizations  up  to  social  pressures  to  improve  the  medication  use 
system  or  risk  losing  business. 

Future  studies  on  risk  factors  and  resource  use  should  be  done  in  large 
administrative  databases.  The  current  study  was  not  able  to  assess  the  causality  of  these 
components.  Resource  use  is  particularly  important  since  preventing  these  events  could 
save  billions  of  dollars  in  the  health  care  system. 


APPENDIX  A 
GERIATRIC  MEDICINE  EXPERT  PANEL  MEMBERS 


Expert  panel  member 

Qualifications 

Samir  Array,  M.D. 

Family  Medicine 

Board  Certified  in  Geriatrics 

Clinical  Assistant  Professor,  Community  Health  and 

Family  Medicine 

James  Bomhard,  M.D. 

Internal  Medicine 

Robert  Curry,  M.D. 

Family  Medicine 

Professor  and  Chair,  Community  Health  and  Family 

Medicine 

Richard  Glock,  M.D. 

Internal  Medicine 

Louis  Laramoyeux,  D.O. 

Family  Medicine 

Harold  Reeder,  M.D. 

Family  Medicine 

Board  Certified  in  Geriatrics 

Clinical  Associate  Professor  and  Program  Director, 

Community  Health  and  Family  Medicine 

Jeffery  Weitzner,  M.D. 

Internal  Medicine 

NOTE:  All  physicians  were  members  of  the  MCO  provider  network 
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APPENDIX  B 
INITIAL  MAILING  AND  INSTRUCTIONS  TO  THE  DELPHI  PANEL 

Dear  Dr.  X 

Thank  you  for  agreeing  to  participate  on  the  Geriatric  Medicine  Expert  Panel  for 
Blue  Cross/Blue  Shield  and  the  University  of  Florida.  Your  expertise  will  be  valuable  in 
looking  at  medication  use  from  an  entirely  different  perspective. 

A  major  problem  in  the  elderly  is  preventable  drug-related  morbidity. 
Preventability  is  defined  as  those  outcomes  that  are  recognizable  and  foreseeable,  and 
where  the  causality  is  identifiable  and  controllable. 

The  attached  survey  contains  detailed  instructions  to  help  you  define  preventable 
drug-related  morbidity  in  the  elderly.  Based  on  past  surveys,  it  should  take  between  60- 
90  minutes  to  complete.  Your  input  is  extremely  important.  Please  return  the  survey  of 
possible  preventable  drug-related  morbidities,  along  with  any  comments,  by  Friday, 
December  17,  1999.  Approximately  7-10  days  after  that,  you  will  receive  a  revised 
survey  based  on  the  comments  of  all  seven  participants. 

You  may  fax  the  survey  back  to  me  at  (352)  392-7782.  Please  call  me  at  (352) 
392-9035  if  you  have  any  questions  or  email  me  at;  faris@cop3.health.ufl.edu. 

Thanks  again  for  participating  in  this  important  task. 
Sincerely, 

Richard  J.  Faris,  M.S.,  R.Ph. 
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Survey  Instructions 

You  will  be  helping  to  define  preventable  drug-related  morbidities  in  the  elderly.  First, 
here  are  a  couple  definitions: 

(1)  Preventable  drug-related  morbidity  (adverse  event,  drug  misadventure,  etc.)  is 
defined  as  a  clinical  outcome  in  which  drug  therapy  has  not  produced  a  reasonable 
intended  result  by  (a)  producing  a  noxious,  unintended  and  undesired  drug  effect,  or 
(b)  by  failing  to  produce  the  intended  effect  within  a  reasonable  time. 

(2)  Preventable  drug-related  morbidity  (a)  results  from  an  unacceptable  quality  of  care 
(e.g.,  failure  to  meet  consensus  guidelines),  or  (b)  occurs  after  a  drug-related 
problem. 

(3)  Preventability:  There  are  four  defining  characteristics  of  preventable  drug-related 
morbidity.  The  drug-related  problem  must  be  recognizable  and  the  likelihood  of  a 
drug-related  morbidity  must  be  foreseeable.  In  addition,  the  cause(s)  of  the  problem 
(and  subsequent  drug-related  morbidity)  must  be  identifiable,  and  those  causes  must 
be  controllable.  Preventable  drug-related  morbidity,  therefore,  results  from 
unrecognized  or  unresolved  drug-related  problems. 


• 


A  drug-related  problem  might  be  recognizable  and  interpretable  by  the  patient  in 
whom  the  event  occurs,  a  lay  caretaker,  or  a  health  professional  (physician, 
pharmacist,  etc.) 

Here  are  the  survey  instructions: 


• 


The  objective  of  this  exercise  is  to  develop  criteria  for  preventable  drug-related 
morbidity  using  the  four  defining  characteristics.  Each  of  the  following  examples 
will  describe  a  drug-related  morbidity  and  additional  information  that  may  or  may  not 
(in  your  judgement)  relate  to  quality  standards  or  describe  a  drug-related  problem. 

In  order  to  evaluate  whether  the  following  examples  are  types  of  preventable  drug- 
related  morbidities,  you  should  read  the  outcomes  and  patterns  of  care  and  answer 
the  following  questions: 

1.  For  most  older  persons,  should  health  professionals  (physicians,  pharmacists, 
etc.)  be  able  to  recognize  significant  problems  in  this  pattern  of  care? 

2.  For  most  older  persons,  should  the  health  professional  be  able  to  foresee  the 
possibility  of  the  outcome,  if  those  problems  were  not  resolved? 

3 .  Should  most  health  professionals  see  how  to  change  the  pattern  of  care  to 
prevent  the  outcome? 

4.  Should  most  health  professionals  actually  change  the  pattern  of  care? 

•     If  you  answer  "Yes"  to  all  four  of  these  questions,  then  this  is  a  type  of 
preventable  drug-related  morbidity  and  you  should  check  the  "Yes"  box. 
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•     If  you  answer  "No"  to  one  or  more  of  these  questions,  then  this  is  not  a  type  of 
preventable  drug-related  morbidity  and  you  should  check  the  "No"  box.  If  you 
answered  "No",  please  specify  why.  You  may  wish  to  describe  whether  there  is 
some  other  element  that  could  be  added  to  the  pattern  of  care  that  would  make  it  a 
stronger,  clearer,  or  less  ambiguous  definition  of  preventable  drug-related 
morbidity.  This  section  is  very  important  to  complete.  Please  be  as  specific  as 
you  possibly  can.  For  example,  if  you  believe  the  pattern  of  care  should  be 
changes  (e.g.,  a  lab  value  should  be  monitored  every  2  months  instead  of  every  3 
months)  or  something  is  missing  (e.g.,  a  duration  of  use  of  a  drug  should  be 
specified)  then  please  write  that  in  the  space  provided.  Also,  if  you  felt  the 
possible  preventable  drug-related  morbidity  did  not  meet  any  one  of  the  four 
criteria,  then  please  state  what  caused  you  to  answer  the  way  you  did  (e.g.,  you 
feel  additional  laboratory  tests  would  be  needed  in  order  to  identify  causality). 

If,  for  some  reason,  you  answer  "Yes"  to  all  four  questions,  but  you  still  think  you 
should  check  the  "No"  box,  please  describe  your  reservations  or  concerns. 

Finally,  there  is  a  space  at  the  end  of  the  list  for  any  additional  preventable  drug- 
related  morbidities  in  older  adults  that  you  feel  are  important  to  add.  Any  additional 
definitions  you  could  provide  based  on  your  knowledge  and  experience  will  be  a 
welcomed  addition  to  the  panel. 

Thank  you  again  for  your  time  and  effort  in  this  endeavor. 


APPENDIX  C 
ROOT  CAUSE  ANALYSIS  EXPERT  PANEL 


Expert  panel  member 

Qualifications 

Birem  Amin,  Pharm.D. 

Family  Practice  Pharmacist 
University  ambulatory  clinic 

Stephen  J.  Clark,  M.D. 

Family  Medicine 
MCO  provider  network 

George  Mayzell,  M.D. 

Internal  Medicine 

Board  Certified  in  Geriatrics 

MCO  regional  medical  director 

Stuart  Millstone,  M.D. 

Pulmonologist 

Board  Certified  in  Pulmonology 

MCO  provider  network 

Mitchell  S.  Rothstein,  M.D. 

Pulmonologist 

Board  Certified  in  Pulmonology 

MCO  provider  network 

Debra  Swartwood,  M.S.,  R.Ph. 

MCO  Clinical  Pharmacy  Manager 
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APPENDIX  D 
FINAL  PREVENTABLE  DRUG-RELATED  MORBIDITY  DEFINITIONS 


1 .  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  depression  and/or  increase  in  dosage  of 
antidepressant 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  depression 

2.  Use  of  long-acting  benzodiazepine  (e.g.,  Librium,  Valium,  Centrax,  Paxipam, 
Dalmane,  Azaene/Tranxene,  etc.) 

2.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Theophylline  toxicity 

This  is  the  pattern  of  care: 

1 .  Use  of  theophylline 

2.  Drug  level  not  done  at  least  every  6  months 

3.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Bipolar  exacerbation  and/or  ER  visit/hospitalization  due  to  bipolar  disorder. 

This  is  the  pattern  of  care: 

1 .  Diagnosis/history  of  bipolar  disorder 

2.  Use  of  lithium 

3.  Drug  level  not  done  at  least  every  three  months 

4.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Major  and/or  minor  hemorrhagic  event 

This  is  the  pattern  of  care: 

1 .  Use  of  IV  heparin 

2.  PTT  not  done  at  least  every  day 

5.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Gastritis  and/or  upper  GI  bleed  and/or  upper  GI  perforation  and/or  GI  ulcers  and 
anemia 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  ulcers  and/or  GI  bleeding 

2.  NSAID  use  for  at  least  1  month 
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6.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  depression  and/or  an  increase  in  the  dosage  of 
antidepressant 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  depression 

2.  Use  of  a  barbiturate  (e.g.,  butalbital) 

7.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  depression  and/or  increased  dosage  of 
antidepressant 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  depression 

2.  Use  of  a  sympatholytic  antihypertensive  (e.g.,  reserpine,  methyldopa, 
clonidine,  etc.) 

8.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Status  epilepticus  and/or  ER  visit/hospitalization  due  to  seizure  activity 

This  is  the  pattern  of  care: 

1 .  Use  of  anticonvulsant  requiring  drug  level  monitoring  (e.g.,  phenytoin, 
carbamazepine,  valproic  acid) 

2.  Drug  level  not  done  upon  initiation  of  therapy  and  at  least  every  6  months 
thereafter 

9.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Lithium  toxicity 

This  is  the  pattern  of  care: 

1 .  Use  of  lithium 

2.  Lithium  level  not  done  monthly  until  stable,  then  every  3  months 

10.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  hyperthyroidism 

This  is  the  pattern  of  care: 

1 .  Use  of  thyroid  or  antithyroid  agent  (e.g.,  levothyroxine,  propylthiouricil,  etc.) 

2.  T4/TSH  not  done  within  6  weeks  after  initiation  of  therapy  and  at  least  every 
12  months  thereafter 

1 1 .  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  a  major/minor  hemorrhagic  event 


107 

This  is  the  pattern  of  care: 

1 .  Warfarin  use 

2.  NSAID  use  (e.g.,  diclofenac,  ibuprofen,  ketoprofen,  etc.) 

3.  INR  not  done  within  10  days 

12.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  hypothyroidism 

This  is  the  pattern  of  care: 

1 .  Lithium  use  for  at  least  6  months 

2.  TSH  not  done  at  least  every  6  months 

13.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Blood  dyscrasias/thrombocytopenia 

This  is  the  pattern  of  care: 

1 .  Use  of  ticlopidine  (Ticlid) 

2.  CBC/platelets  not  done  at  baseline,  within  2  weeks  of  start  of  therapy  and 
within  2  months 

14.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Rebound  congestion 

This  is  the  pattern  of  care: 

1 .    Use  of  long-acting  nasal  spray  (e.g.,  oxymetazoline)  for  more  than  3  days 

15.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  urinary  retention 

This  is  the  pattern  of  care: 

1 .  Diagnosis/history  of  bladder  atony  due  to  diabetes 

2.  Use  of  imipramine 

16.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  respiratory  failure 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  severe  COPD 

2.  Use  of  medium  to  long- acting  benzodiazepines 

17.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  urinary  retention 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  benign  prostatic  hypertrophy  (BPH) 

2.  Use  of  an  anticholinergic  agent 
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18.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  liver  toxicity 

This  is  the  pattern  of  care: 

1.  Use  of  troglitazone  (Rezulin) 

2.  Liver  function  tests  not  done  at  baseline  and  at  least  monthly  for  the  first  8 
months  of  therapy  and  at  least  every  2  months  for  the  remainder  of  the  first 
year 

19.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  congestive  heart  failure  and/or  fluid  overload 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  high  blood  pressure  (over  140/90)  and/or  congestive 
heart  failure 

2.  NSAID  use  for  at  least  3  months 

20.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  for  hypokalemia 

This  is  the  pattern  of  care: 

1 .  Use  of  potassium- wasting  diuretic  (e.g.,  hydrochlorothiazide,  etc.) 

2.  No  concurrent  use  of  potassium  supplement 

3.  Electrolytes  not  checked  at  least  every  2  months  initially,  then  every  6  months 

21 .  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Anticonvulsant  drug  toxicity 

This  is  the  pattern  of  care: 

1 .  Use  of  an  anticonvulsant  requiring  drug  level  monitoring  (e.g.,  phenytoin, 
carbamazepine,  valproic  acid) 

2.  Drug  level  not  done  at  least  every  6  months 

22.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  hypothyroidism 

This  is  the  pattern  of  care: 

1.  Use  of  thyroid  or  antithyroid  agent  (e.g.,  levothyroxine,  propylthiouricil,  etc.) 

2.  T4/TSH  not  done  before  therapy  starts  and  at  least  every  12  months  thereafter 

23.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Hospitalization/ER  visit  due  to  worsening  renal  impairment  and/or  acute  renal 
failure  and  or  renal  insufficiency 
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This  is  the  pattern  of  care: 

1 .  Diagnosis/history  of  moderate  to  severe  renal  impairment/  history  of  kidney 
disease 

2.  Use  of  a  select  urinary  antiinfective  agent  (nalidixic  acid,  nitrofurantoin  or 
methenamine  complexes) 

3.  BUN/serum  creatinine  not  done  within  30  days  of  initiation  of  therapy  and  at 
least  every  6  months 

24.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Aminoglycoside  toxicity  (acute  renal  failure  and/or  renal  insufficiency  and/or 
vestibular  damage  and/or  auditory  damage) 

This  is  the  pattern  of  care: 

1 .  Use  of  an  aminoglycoside 

2.  Serum  creatinine  not  done  before  and  after  therapy  (and  if  therapy  longer  than 
7  days,  not  done  at  least  every  7  days) 

3 .  At  least  one  drug  level  not  done 

25.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

COPD  exacerbation  and/or  ER  visit/hospitalization  due  to  COPD 

This  is  the  pattern  of  care: 

1 .  Diagnosis/history  of  COPD 

2.  Use  of  a  beta-blocker  (e.g.,  propranolol,  etc.) 

26.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Hypertension/tachycardia 

This  is  the  pattern  of  care: 

1 .  History  of  hypertension 

2.  Use  of  decongestants 

27.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  worsening  renal  impairment  and/or  acute  renal 
failure  and/or  renal  insufficiency 

This  is  the  pattern  of  care: 

1 .  Diagnosis/history  of  moderate  to  severe  renal  impairment  and/or  history  of 
kidney  disease 

2.  Use  of  tetracycline 

3.  BUN/serum  creatinine  not  done  within  30  days  of  initiation  of  therapy  and  at 
least  every  6  months 

28.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  hyperkalemia 
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This  is  the  pattern  of  care: 

1 .  Use  of  ACE  inhibitor 

2.  Electrolytes/CBC  not  done  at  least  every  6  months 

29.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Blood  dyscrasias  and/or  hyponatremia  and/or  excessive  water  retention  and/or 
syndrome  of  inappropriate  antidiuretic  hormone  (SIADH) 

This  is  the  pattern  of  care: 

1 .  Use  of  carbamazepine 

2.  Electrolytes/CBC  not  done  before  therapy  initiated,  at  least  weekly  during  the 
first  month  of  therapy,  at  least  monthly  during  the  next  5  months  of  therapy, 
and  at  least  every  6  months  thereafter 

30.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  renal  failure  and/or  renal  insufficiency 

This  is  the  pattern  of  care: 

1 .  Use  of  lithium 

2.  BUN/serum  creatinine  not  done  at  least  every  3  months 

3 1 .  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Digoxin  toxicity 

This  is  the  pattern  of  care: 

1.  Use  of  digoxin 

2.  BUN/serum  creatinine  not  done  at  least  every  6  months 

3.  Digoxin  level  not  done  at  least  every  6  months 

32.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Gastritis  and/or  upper  GI  bleeds  and/or  GI  perforations  and/or  GI  ulcers  and 
anemia 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  ulcers  and/or  gastrointestinal  bleeding 

2.  Use  of  an  oral  corticosteroid  (e.g.,  prednisone)  for  at  least  3  months 

33.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  congestive  heart  failure  and/or  heart  block 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  congestive  heart  failure  with  heart  block  or  advanced 
bradycardia 

2.  Use  of  digoxin 
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34.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  renal  failure  and/or  renal  insufficiency 

This  is  the  pattern  of  care: 

1 .  Use  of  an  ACE  inhibitor 

2.  BUN/serum  creatinine  not  done  at  initiation  of  therapy  and  at  least  every  3-6 
months  thereafter 

35.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Major  and/or  minor  hemorrhagic  event 

This  is  the  pattern  of  care: 

1 .  Use  of  warfarin 

2.  INR  not  done  before  therapy  starts  and  at  least  every  month  thereafter 

36.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Secondary  myocardial  infarction 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  myocardial  infarction 

2.  No  use  of  ASA  and/or  beta  blocker  (e.g.,  metoprolol) 

37.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Blood  dyscrasias 

This  is  the  pattern  of  care: 

1 .  Concurrent  use  of  trimethoprim/sulfamethoxazole  (Bactrim/Septra)  and 
methotrexate 

2.  WBC/platelets/CBC  not  done  at  least  every  4  weeks 

38.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  a  major/minor  hemorrhagic  event 

This  is  the  pattern  of  care: 

1 .  Warfarin  use 

2.  Antibiotic  use  (e.g.,  Bactrim,  etc.) 

3.  PT  not  done  within  5  days 

39.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Gastritis  and/or  upper  GI  bleed  and/or  GI  perforation  and/or  GI  ulcer  and  anemia 

This  is  the  pattern  of  care: 

Use  of  2  or  more  NSAIDS  concurrently  for  at  least  2  weeks 

40.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  renal  failure  and/or  renal  insufficiency 
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This  is  the  pattern  of  care: 

1 .  Use  of  allopurinol 

2.  BUN/serum  creatinine  not  done  at  least  every  6  months 

41.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  depression  and/or  increase  in  dosage  of 
antidepressant 

This  is  the  pattern  of  care: 

1 .  History/diagnosis  of  depression 

2.  Use  of  moderate  to  high  lipophilic  beta-adrenergic  blocking  agent  (e.g. 
propranolol,  pindolol,  etc.) 

42.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Falls  and/or  bone  fracture  and/or  bone  break 

This  is  the  pattern  of  care: 

1 .  Use  of  muscle  relaxants 

2.  Use  of  cough  codeine-containing  cough  suppressants 

43.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Patient  fall  and  hip  fracture 

This  is  the  pattern  of  care: 

1 .  Patient  on  alpha  blocker 

2.  Standing  BP  not  checked  within  2  months  of  initiation  of  therapy 

44.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Fall  and/or  hip  fracture  and/or  other  bone  fracture  and/or  bone  break 

This  is  the  pattern  of  care: 

Use  of  long  half-life  hypnotic/anxiolytic  (e.g.,  flurazepam,  diazepam, 
chlordiazepoxide,  etc.) 

45.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  congestive  heart  failure 

This  is  the  pattern  of  care: 

1 .  Diagnosis/history  of  congestive  heart  failure 

2.  Not  on  an  ACE  inhibitor  (e.g.,  captopril,  enalapril,  etc.) 

46.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Acute  renal  failure  and/or  renal  insufficiency 
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This  is  the  pattern  of  care: 

1 .  NSAID  use  for  at  least  3  months 

2.  BUN/serum  creatinine  not  done  at  least  every  3  months 

47.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

ER  visit/hospitalization  due  to  congestive  heart  failure 

This  is  the  pattern  of  care: 

1 .  History /diagnosis  of  congestive  heart  failure 

2.  Use  of  an  antiarrhythmic  agent  (e.g.,  disopyramide,  procainamide,  etc.) 

48.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Asthma  exacerbation  and/or  status  asthmaticus  and/or  ER  visit/hospitalization  due 
to  asthma 

This  is  the  pattern  of  care: 

1 .  Diagnosis  of  moderate  to  severe  asthma 

2.  Use  of  a  bronchodilator 

3.  No  use  of  maintenance  corticosteroid  (e.g.,  beclomethasone,  etc.) 

49.  This  outcome  has  occurred  after  the  pattern  of  care  below: 

Fall  and/or  hip  fracture  and/or  other  bone  fracture  and/or  bone  break. 

This  is  the  pattern  of  care: 

1 .    Use  of  tricyclic  antidepressant  (e.g.,  amitriptyline,  doxepin,  imipramine.  etc.) 


APPENDIX  E 
MNEMONICS  FOR  CONSENSUS-APPROVED  DEFINITIONS 


Definition3 

Mnemonic 

Definition 

Mnemonic3 

1 

Dep/Hx-Benzo 

26 

HTN/Hx-Decon 

2 

Theo/DrugLvl 

27 

ARnF/Hx-TCN-BUNCr 

3 

BP/Hx-Li-DrugLvl 

28 

HyprK/ACEI-EleCBC 

4 

Hmr/hep-NoLab 

29 

Bldys/Cbz-EleCBC 

5 

GI/Hx-NSAID 

30 

ARnF/Li-BUNCr 

6 

Dep/Hx-Barb 

31 

Dig/Dig-BUNCr-DrugLvl 

7 

Dep/Hx-Symp 

32 

GI/Hx-OCort 

8 

Seiz/Antconv-DrugLvl 

33 

CHF/Hx-Dig 

9 

Li/Li-DrugLvl 

34 

ARnF/ACEI-BUNCr 

10 

Hyprth/Thy-T4TSH 

35 

Hmr/War-INR 

11 

Hmr/War-NSAID-INR 

36 

MI/Hx-ASA-BB 

12 

Hypoth/Li-TSH 

37 

Bldys/TMPSMX-NoLab 

13 

Bldys/Tic-CBCP 

38 

Hmr/War-ABx-NoLab 

14 

RC/Lan 

39 

GI/NSAID 

15 

AUR/Hx-Imip 

40 

ARnF/Allp-BUNCr 

16 

ARsF/COPD-Benzo 

41 

Dep/Hx-BB 

17 

AUR/Hx-Antic 

42 

FB/MscRelax 

18 

LvrTox/Rez-LvrTest 

43 

FB/AB-SBP 

19 

CHF/Hx-NSAID 

44 

FB/SedHyp 

20 

HypoK/Kwd-NoK-Elec 

45 

CHF/Hx-ACEI 

21 

Antconv/Anticonv-DrugLvl 

46 

ARnF/NSAID-BUNCr 

22 

Hypoth/Thy-T4TSH 

47 

CHF/Hx-AArhy 

23 

ARnF/Hx-SUI-BUNCr 

48 

Asth/Hx-Bdilat-ICort 

24 

Amgly/Amgly-SCr-DrugLvl 

49 

FB/TCA 

25 

COPD/Hx-BB 

Definition  number  from  Appendix  D 
NOTE:  Alphabetical  list  of  mnemonics  in  Appendix  F 
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APPENDIX  F 
ALPHABETICAL  MNEMONIC  LIST 


Mnemonic 

Outcome  or  pattern  of  care 

AArhy 

antiarrhythmic 

AB 

Alpha  blocker 

ABx 

Antibiotics 

ACEI 

ACE  Inhibitor 

Allp 

Allopurinol 

Amgly 

Aminoglycoside  use  or  toxicity 

Antic 

Anticholinergic 

Anticonv 

Anticonvulsant  use  or  toxicity 

ARnF 

Acute  renal  failure 

ARsF 

Acute  respiratory  failure 

ASA 

Aspirin 

Asth 

Asthma 

AUR 

Acute  urinary  retention 

Barb 

Barbiturate 

BB 

Beta-blocker 

Bdilat 

Bronchodilator 

Benzo 

Benzodiazepine 

Bldys 

Blood  dyscrasias 

Bp 

Bipolar 

BUNCr 

BUN  and  serum  creatinine 

CBCP 

CBC  and  platelets 

Cbz 

Carbamazepine 

CHF 

Congestive  heart  failure 

COPD 

COPD 

Decon 

Decongestant 

Dep 

Depression 

Dig 

Digoxin  use  or  toxicity 

DrugLvl 

Drug  level 

Elec 

Electrolytes 

EleCBC 

Electrolytes  and  CBC 

FB 

Fall  or  bone  break 

GI 

Gastrointestinal 

Hep 

Heparin 

Hmr 

Hemorrhage 

HTN 

Hypertension 
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Mnemonic 

Outcome  or  pattern  of  care 

Hx 

History 

HypoK 

Hypokalemia 

Hypoth 

Hypothyroidism 

HyprK 

Hyperkalemia 

Hyprth 

Hyperthyroidism 

ICort 

Inhaled  corticosteroid 

Imip 

Imipramine 

INR 

INR  level 

Kwd 

Potassium  wasting  diuretic 

Lan 

Long-acting  nasal  spray 

Li 

Lithium  use  or  toxicity 

LvlTox 

Liver  toxicity 

LvrTest 

Liver  function  tests 

MI 

Myocardial  infarction 

MscRelax 

Muscle  relaxants 

NoK 

No  potassium  supplement 

NoLab 

No  laboratory  monitoring  done 

NSAID 

Non-steroidal  anti-inflammatory  drug 

Ocort 

Oral  corticosteroids 

RC 

Rebound  congestion 

Rez 

Rezulin 

SBP 

Standing  blood  pressure 

SCr 

Serum  creatinine 

SedHyp 

Sedative/hypnotic 

Seiz 

Seizure 

SUI 

Select  urinary  antiinfectives 

Symp 

Sympatholytic  antihypertensive 

T4TSH 

T4/TSH  level 

TCA 

Tricyclic  antidepressant 

TCN 

Tetracycline 

Theo 

Theophylline  toxicity 

Thv 

Thyroid  or  antithyroid  agent 

Tic 

Ticlodipine 

TMPSMX 

Trimethoprim/sulfamethoxazole 

TSH 

TSH  level 

War 

Warfarin 

APPENDIX  G 
NUMBER  OF  PATIENTS  WITH  EACH  PDRM  DEFECTION 


Definition3 

Mnemonic" 

Instances  of  PDRMC 

Percent  of  total  PDRMC 

1 

Dep/Hx-Benzo 

56 

4.1% 

2 

Theo/DrugLvl 

0 

0.0% 

3 

BP/Hx-Li-DrugLvl 

0 

0.0% 

4 

Hmr/hep-NoLab 

0 

0.0% 

5 

GI/Hx-NSAID 

63 

4.6% 

6 

Dep/Hx-Barb 

1 

0.1% 

7 

Dep/Hx-Symp 

35 

2.6% 

8 

Seiz/Antconv-DrugLvl 

10 

0.7% 

9 

Li/Li-DrugLvl 

0 

0.0% 

10 

Hyprth/Thy-T4TSH 

1 

0.1% 

11 

Hmr/War-NSAID-INR 

0 

0.0% 

12 

Hypoth/Li-TSH 

0 

0.0% 

13 

Bldys/Tic-CBCP 

0 

0.0% 

14 

RC/Lan 

0 

0.0% 

15 

AUR/Hx-Imip 

0 

0.0% 

16 

ARsF/COPD-Benzo 

19 

1 .4% 

17 

AUR/Hx-Antic 

4 

0.3% 

18 

LvrTox/Rez-LvrTest 

0 

0.0% 

19 

CHF/Hx-NSAID 

80 

5.9% 

20 

HypoK/Kwd-NoK-Elec 

44 

3.2% 

21 

Antconv/Anticonv-DrugLvl 

0 

0.0% 

22 

Hypoth/Thy-T4TSH 

103 

7.6% 

23 

ARnF/Hx-SUI-BUNCr 

0 

0.0% 

24 

Amgly/Amgly-SCr-DrugLvl 

0 

0.0% 

25 

COPD/Hx-BB 

54 

4.0% 

26 

HTN/Hx-Decon 

0 

0.0% 

27 

ARnF/Hx-TCN-BUNCr 

0 

0.0% 

28 

HyprK/ACEI-EleCBC 

14 

1.0% 

29 

Bldys/Cbz-EleCBC 

0 

0.0% 

30 

ARnF/Li-BUNCr 

0 

0.0% 

31 

Dig/Dig-BUNCr-DrugLvl 

1 

0.1% 

32 

GI/Hx-OCort 

38 

2.8% 

33 

CHF/Hx-Dig 

184 

13.5% 

34 

ARnF/ACEI-BUNCr 

1 

0.1% 
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Definition3 

Mnemonic6 

Instances  of  PDRMC 

Percent  of  total  PDRMC 

35 

Hmr/War-INR 

3 

0.2% 

36 

MI/Hx-ASA-BB 

54 

4.0% 

37 

Bldys/TMPSMX-NoLab 

0 

0.0% 

38 

Hmr/War-ABx-NoLab 

0 

0.0% 

39 

GI/NSAID 

129 

9.5% 

40 

ARnF/Allp-BUNCr 

1 

0.1% 

41 

Dep/Hx-BB 

15 

1.1% 

42 

FB/MscRelax 

1 

0.1% 

43 

FB/AB-SBP 

0 

0.0% 

44 

FB/SedHyp 

42 

3.1% 

45 

CHF/Hx-ACEI 

270 

19.9% 

46 

ARnF/NSAID-BUNCr 

2 

0.1% 

47 

CHF/Hx-AArhy 

37 

2.7% 

48 

Asth/Hx-Bdilat-ICort 

89 

6.5% 

49 

FB/TCA 

8 

0.6% 

"Definition  number  matches  Appendix  D 
''Mnemonics  in  appendices  E  and  F 
CPDRM:  preventable  drug-related  morbidity 
NOTE:  1,359  instances  detected  in  966  patients 


APPENDIX  H 
PDRM,  PATTERN  OF  CARE  ONLY,  AND  OUTCOME  ONLY 


Definition3 

Mnemonic" 

PDRMC 

Pattern  onlyd 

Outcome  only 

1 

Dep/Hx-Benzo 

56 

1 

382 

2 

Theo/DrugLvl 

0 

83 

2 

3 

BP/Hx-Li-DrugLvl 

0 

0 

43 

4 

Hmr/hep-NoLab 

0 

0 

1571 

5 

GI/Hx-NSAID 

63 

1 

760 

6 

Dep/Hx-Barb 

1 

0 

512 

7 

Dep/Hx-Symp 

35 

0 

426 

8 

Seiz/Antconv-DrugLvl 

10 

41 

164 

9 

Li/Li-DrugLvl 

0 

3 

76 

10 

Hyprth/Thy-T4TSH 

1 

319 

41 

11 

Hmr/War-NSAID-INR 

0 

2 

1570 

12 

Hypoth/Li-TSH 

0 

.1 

712 

13 

Bldys/Tic-CBCP 

0 

6 

99 

14 

RC/Lan 

0 

0 

61 

15 

AUR/Hx-Imip 

0 

5 

169 

16 

ARsF/COPD-Benzo 

19 

173 

157 

17 

AUR/Hx-Antic 

4 

31 

163 

18 

LvrTox/Rez-LvrTest 

0 

56 

66 

19 

CHF/Hx-NSAID 

80 

814 

680 

20 

HypoK/Kwd-NoK-Elec 

44 

74, S 

318 

21 

Antconv/Anticonv-DrugLvl 

0 

71 

0 

22 

Hypoth/Thy-T4TSH 

103 

175 

566 

23 

ARnF/Hx-SUI-BUNCr 

0 

0 

250 

24 

Amgly/Amgly-SCr-DrugLvl 

0 

0 

350 

25 

COPD/Hx-BB 

54 

1 

1038 

26 

HTN/Hx-Decon 

0 

1 

4237 

27 

ARnF/Hx-TCN-BUNCr 

0 

0 

250 

28 

HyprK/ACEI-EleCBC 

14 

465 

335 

29 

Bldys/Cbz-EleCBC 

0 

14 

440 

30 

ARnF/Li-BUNCr 

0 

3 

250 

31 

Dig/Dig-BUNCr-DrugLvl 

1 

97 

80 

32 

GI/Hx-OCort 

38 

1 

854 

33 

CHF/Hx-Dig 

184 

50 

653 

34 

ARnF/ACEI-BUNCr 

Jj 

314 

248 
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Definition" 

Mnemonic" 

PDRMC 

Pattern  onlyd 

Outcome  only 

35 

Hmr/War-INR 

3 

11 

1564 

36 

MI/Hx-ASA-BB 

54 

44 

63 

37 

Bldys/TMPSMX-NoLab 

0 

171 

10 

38 

Hmr/War-ABx-NoLab 

0 

1 

1568 

39 

GI/NSAID 

129 

1549 

760 

40 

ARnF/Allp-BUNCr 

1 

19 

249 

41 

Dep/Hx-BB 

15 

1 

471 

42 

FB/MscRelax 

1 

5 

562 

43 

FB/AB-SBP 

0 

0 

175 

44 

FB/SedHyp 

42 

878 

468 

45 

CHF/Hx-ACEI 

270 

299 

303 

46 

ARnF/NSAID-BUNCr 

2 

491 

248 

47 

CHF/Hx-AArhy 

37 

27 

745 

48 

Asth/Hx-Bdilat-ICort 

89 

0 

173 

4l) 

FB.'TCA 

8 

134 

547 

"Definition  number  matches  Appendix  D 

''Mnemonics  in  appendices  E  and  F 

CPDRM:  preventable  drug-related  morbidity 

dPattern  only  is  before  adjustment  in  the  number  of  days  for  actual  PDRM  events. 


APPENDIX  I 
CUT-OFF  DAYS  FOR  PDRM  DEFINITIONS  BASED  ON  METHOD 


Definition3'" 

Mean  =  1.65  SDC 

Linear  interpolationd 

Exponential6 

1  (Dep/Hx-Benzo) 

339.98 

338 

316.68 

5  (GI/Hx-NSAID) 

255.75 

333 

394.75 

7  (Dep/Hx-Symp) 

279.13 

311 

302.90 

8  (Seiz/Antconv-DrugLvl) 

264.73 

242 

117.43 

16(ARsF/COPD-Benzo) 

326.12 

318 

305.83 

19(CHF/Hx-NSAID) 

352.87 

368 

301.04 

20  (HypoK/Kwd-NoK-Elec) 

311.77 

304 

300.05 

22  (Hypoth/Thy-T4TSH) 

377.40 

385 

228.93 

25  (COPD/Hx-BB) 

335.77 

376 

461.34 

28  (HyprK/ACEI-EleCBC) 

393.64 

372 

209.16 

32  (GI/Hx-Ocort) 

257.34 

324 

352.93 

33  (CHF/Hx-Dig) 

352.59 

385 

299.66 

36  (MI/Hx-ASA-BB) 

82.04 

91 

235.34 

39  (GI/NSAID) 

367.08 

376 

246.43 

41  (Dep/Hx-BB) 

317.55 

302 

299.15 

44  (FB/SedHyp) 

375.71 

364 

193.43 

45  (CHF/Hx-ACEI) 

333.42 

374 

403.14 

47  (CHF/Hx-AArhy) 

315.95 

327 

327.23 

48  (Asth/Hx-Bdilat-ICort) 

379.57 

393 

328.33 

"Definition  number  matches  Appendix  D 

''Mnemonic  matches  appendices  E  and  F 

^Mean  =  1 .65  SD:  mean  plus  1 .65  times  the  standard  deviation  for  each  definition 

dLinear  interpolation:  linear  interpolation  of  the  95%  cut-off  value 

Exponential:  cumulative  exponential  distribution,  95%  cut-off  point 
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APPENDIX  J 
PATTERN  OF  CARE  ONLY  MEASURES  BASED  ON  95%  CUT-OFF  VALUE 


Definition3-" 

PDRMC  events 

All  pattern  of 
care-only  patients 

Pattern  of  care 

patients  after  95% 

cut-off  value 

1  (Dep/Hx-Benzo) 

56 

1 

0 

5  (GI/Hx-NSAID) 

63 

1 

1 

7  (Dep/Hx-Symp) 

35 

0 

0 

8  (Seiz/Antconv-DrugLvl) 

10 

41 

35 

16(ARsF/COPD-Benzo) 

19 

173 

53 

19(CHF/Hx-NSAID) 

80 

814 

139 

20  (HypoK/Kwd-NoK-Elec) 

44 

748 

365 

22  (Hypoth/Thy-T4TSH) 

103 

175 

127 

25  (COPD/Hx-BB) 

54 

1 

0 

28  (HyprK/ACEI-EleCBC) 

14 

465 

360 

32  (GI/Hx-Ocort) 

38 

1 

0 

33  (CHF/Hx-Dig) 

184 

50 

9 

36  (MI/Hx-ASA-BB) 

54 

44 

33 

39  (GI/NSAID) 

129 

1549 

929 

41  (Dep/Hx-BB) 

15 

1 

0 

44  (FB/SedHyp) 

42 

878 

655 

45  (CHF/Hx-ACEI) 

270 

299 

90 

47  (CHF/Hx-AArhy) 

37 

27 

6 

48  (Asth/Hx-Bdilat-ICort) 

89 

0 

0 

"Definition  number  matches  Appendix  D 
''Mnemonic  matches  appendices  E  and  F 
CPDRM:  preventable  drug-related  morbidity 
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APPENDIX  K 
CONGESTIVE  HEART  FAILURE  CAUSE-AND-EFFECT  DIAGRAM 
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APPENDIX  L 
GASTROINTESTINAL  CAUSE-AND-EFFECT  DIAGRAM 
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APPENDIX  M 
MYOCARDIAL  INFARCTION  CAUSE-AND-EFFECT  DIAGRAM 
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APPENDIX  N 
MISCELLANEOUS  CAUSE-AND-EFFECT  DIAGRAM 
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APPENDIX  O 
CONSULTANT  ANALYSIS  OF  ROOT  CAUSE  ANALYSIS 


Reaching  Consensus  on  Quality  Indicators:  A  Meeting  with  Blue  Cross/Blue  Shield 

Providers 


Introduction 

On  Thursday,  August  3 1 ,  2000  three  physicians  and  one  pharmacist  provider  met 
with  three  (managed  care  organization)  employees  and  the  research  team  to  review  a  set 
of  quality  indicators.  The  meeting  took  place  in  a  (managed  care  organization) 
conference  room.  The  purpose  of  the  meeting  was  to  reach  consensus  on  the  root  causes 
of  problems  inferred  by  the  indicators. 

The  following  report  was  prepared  to  provide  an  overview  of  the  meeting's 
process.  That  is,  how  did  the  participants  appear  to  react  to  the  to  the  questions  posed 
and  was  consensus  reached  on  root  causes.  A  brief  description  of  each  provider 
participant  is  provided.  This  is  followed  by  a  description  of  the  meeting  process.  The 
final  section  of  this  report  provides  some  general  comments  and  suggestions. 

The  Participants 

Stephen  Clark: 

Dr.  Clark  practices  family  medicine  in  Jacksonville,  Florida.  He  arrived  wearing 
a  suit,  but  took  off  the  jacket  after  greeting  the  rest  of  the  group.  Dr.  Clark  appeared  to  be 
engaged  in  the  process  throughout  the  meeting.  He  had  no  problem  in  responding  when 
asked,  and  he  seemed  to  jump  in  when  he  felt  that  he  had  a  contribution.  While  he 
occasionally  prefaced  his  remarks  with  the  word  "gently"  (in  deference  to  the  (managed 
care  organization)  participants),  he  was  confident  in  his  remarks.  Dr.  Clark  seemed  to 
have  an  interest  in  the  bottom  line.  That  is,  several  of  his  remarks  were  prefaced  by  a 
comment  about  what  (the  managed  care  organization)  wanted  or  how  the  item  under 
discussion  might  affect  (the  managed  care  organization  's)  bottom  line.  This  may 
indicate  that  his  comments  were  directed  at  that  endpoint,  not  necessarily  at  how  the 
indicators  might  help  his  individual  practice. 

Mitchell  Rolhstein: 

Dr.  Rothstein  practices  pulmonary  and  critical  care  medicine  in  Orange  Park. 
Florida.  He  arrived  wearing  a  long  sleeve  shirt  and  dress  slacks,  but  no  tie.  Dr.  Rothstein 
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appeared  to  be  very  actively  engaged  in  the  meeting  process  at  the  onset.  In  the 
beginning  he  was  observed  to  be  studying  the  handouts  and  taking  notes.  He  asked  for 
clarification  of  terms  and  appeared  to  process  the  responses.  Like  Dr.  Clark,  Dr. 
Rothstein  seemed  to  have  a  ready  answer  when  asked.  In  turn,  he  would  ask  for 
clarification  when  he  needed  it.  Towards  the  end  of  the  meeting  while  he  did  still 
participate,  his  attention  seemed  to  waiver,  and  at  the  end  of  the  meeting  he  left  his 
handouts  on  the  table. 

Stuart  Millstone: 

Dr.  Millstone  practices  pulmonary,  internal,  and  critical  care  medicine  in  Orange 
Park,  Florida.  It  appears  that  Dr.  Rothstein  is  his  partner.  He  arrived  thirty  minutes  late 
wearing  a  long  sleeved  shirt  and  dress  slacks.  While  he  did  arrive  after  the  initial 
handout  of  material,  he  did  arrive  in  time  to  hear  the  explanation  of  how  the  root  cause 
discussion  would  take  place.  While  Dr.  Millstone  did  participate  when  asked  he 
appeared  to  be  the  least  engaged  of  the  three  physicians.  On  at  least  two  times  he 
engaged  Drs.  Rothstein  and  Mayzell  in  side  conversations.  He  did  not  sit  up  to  the  table, 
his  arms  were  frequently  crossed,  and  he  was  observed  tapping  his  fingers  on  his  chair 
frequently.  At  one  point  he  left  the  room  after  receiving  a  page. 

Birem  Amin: 

Dr.  Amin  is  a  Family  Practice  Pharmacist  at  Shands  Jacksonville.  Dr.  Amin  also 
arrived  wearing  a  long  sleeved  shirt  and  dress  slacks.  Of  all  of  the  participants  Dr.  Amin 
had  the  least  to  say.  He  was  observed  to  make  only  six  comments  throughout  the 
meeting.  Most  of  those  comments  were  made  only  because  he  was  directly  asked  a 
question.  The  lack  of  participation  may  be  due  to  a  perceived  lack  of  "rank"  in  the  room. 
That  is,  the  other  participants  were  physicians,  employees  of  the  host  organization,  or  part 
of  a  research  team. 

The  Meeting  Process 
Introduction: 

The  meeting  began  at  5:30  with  some  informal  introductions  and  the  offer  of 
sandwiches  and  drinks.  After  a  formal  introduction  of  all  in  the  room,  Dr.  Mayzell 
briefly  introduced  the  project  and  turned  the  floor  over  to  Richard  Faris.  (Mr.  Faris) 
made  a  brief  presentation  and  handed  out  the  material  to  be  used  during  discussion. 
There  was  some  discussion  of  the  method  used  for  obtaining  the  indicator  results,  and 
time  allowed  for  "studying"  the  handouts. 

Root  Causes: 

At  6: 10  the  meeting  was  turned  over  to  Jill  Sanborne,  an  employee  of  (the 
managed  care  organization).  Jill's  role  was  to  facilitate  this  portion  of  the  meeting.  She 
was  to  direct  the  discussion  to  finding  the  root  causes  of  indicator  problems  using  the 
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fishbone  technique.  Jill  began  the  discussion  with  a  question  for  the  participants.  This 
first  question  was  greeted  with  silence.  The  participants  did  not  seem  to  understand  the 
goal  of  the  process/questions.  Dr.  Mayzell  attempted  to  help,  and  while  there  was  a  brief 
discussion  among  the  participants,  there  was  not  a  clear  understanding  of  the  process  that 
they  were  embarking  upon. 

Ms.  Sanborne  moved  forward  and  attempted  to  describe  the  specific  process  that 
they  (the  participants)  would  follow  for  the  remainder  of  the  meeting.  Again  there 
seemed  to  be  a  lack  of  understanding  among  the  participants.  Specifically,  Ms.  Sanborne 
asked  the  participants  to  think  in  broad  terms  (i.e.,  population)  when  responding.  This 
seemed  to  confuse  the  participants.  Also,  there  was  no  clear  presentation  or 
understanding  of  how  to  use  the  fishbone. 

Sensing  this  lack  of  understanding  both  Dr.  Hepler  and  Dr.  Mayzell  attempted  to 
make  the  process  more  concrete.  However,  it  was  not  until  a  specific  group  of  indicators 
(those  for  CHF)  was  chosen  that  the  participants  began  to  respond.  For  one  round  of 
responses  the  participants  followed  one  another  as  the  process  would  indicate,  but  after 
that  one  round  the  participants  jumped  in  whenever  they  had  a  thought/idea. 

While  the  fishbone  diagram  was  available  (as  an  overhead  transparency)  and 
used,  it  did  not  seem  that  the  participants  fully  understood  how  the  available  categories 
and  outcome  worked  together.  In  fact,  at  some  points  (Mr.  Paris)  was  placing  responses 
on  categories  without  participant  input. 

Closing: 

There  was  a  definite  focus  (throughout  the  time)  on  ending  the  meeting  as 
scheduled.  At  the  conclusion  of  the  meeting  the  participants  left  quickly  without  much 
socializing. 


Reflections 
Root  Causes: 

Two  questions  were  raised  during  this  portion  of  the  meeting:  1)  is  there  any 
utility  for  you  (i.e.,  the  participant)  in  this  data?  and  2)  did  the  participants  come  to 
consensus  on  the  root  causes  associated  with  the  indicators?  Only  Drs.  Rothstein  and 
Clark  responded  immediately  and  not  in  the  affirmative  to  question  one.  There  was  a 
small  discussion  of  how  to  make  the  information  useful,  but  I  did  not  sense  that  the 
participants  would  dig  in  and  use  the  information  provided  by  the  indicators.  Comments 
made  about  the  reports  received  from  other  organizations  (e.g.,  PCS)  seem  to  support  a 
lack  of  interest  in  another  report. 

The  question  of  reaching  consensus  is  harder  to  directly  answer.  While  there  was 
a  good  deal  of  discussion  and  response  to  the  indicators  presented,  I  do  not  believe  that  a 
general  consensus  on  root  causes  was  reached.  There  did  seem  to  be  consensus  and  a 
good  number  of  responses  on  some  specific  root  causes  for  specific  indicators.  However, 
the  participants  seemed  to  have  a  difficult  time  thinking  in  terms  of  a  population.  But  is 
this  really  surprising  given  their  training  to  focus  on  the  problem(s)  presented  by  an 
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individual  patient?  Their  task  is  to  solve  a  very  specific  problem  for  the  individual  in 
front  of  them,  not  the  population  of  Duval  County.  Comparing  the  responses  to 
individual  indicators  may  begin  to  show  patterns  of  root  causes,  but  I  would  suggest  that 
these  be  confirmed  by  the  participants  or  others  like  them. 

In  regards  to  the  root  causes  discussion  process,  I  have  some  suggestions.  First, 
do  not  use  an  overhead  transparency  for  the  fishbone  diagram.  The  machine  is 
distracting  and  dim  lights  may  impede  discussion.  Second,  given  the  same  participant 
group,  I  would  begin  the  process  with  a  specific  indicator.  Use  the  specific  as  an 
example  and  then  move  to  the  general  population  based  questions.  Also,  be  careful  of 
making  decisions  for  the  participants  (e.g.,  placing  the  responses  in  categories). 

In  General: 

The  participant  group  seemed  to  be  a  good  match.  The  pharmacist  was  the  only 
individual  to  provide  very  little  input.  None  of  the  physician  participants  seemed  to  be 
intimidated  by  the  others  and  all  had  no  problem  in  "arguing"  their  point  and  suggesting 
that  physicians  are  at  the  root  of  some  of  the  problems.  However,  at  least  one  participant 
(Dr.  Clark)  did  seem  to  focus  some  of  his  responses  on  what  would  be  best  for  (the 
managed  care  organization),  not  for  his  own  practice.  Also,  some  of  the  joking  (who 
will  have  lunch  with  who)  between  (the  managed  care  organization)  personnel  and  the 
participants  seemed  to  indicate  that  the  participants  understood  who  had  the  "power"  in 
the  room.  That  is,  (the  managed  care  organization)  (who  may  be  a  major  payer  in  their 
practice)  asked  them  to  participate  in  a  meeting  to  be  held  in  their  conference  room  with 
their  personnel  present.  While  I  understand  that  (the  managed  care  organization)  is  a 
sponsor  of  the  research,  holding  the  meeting  at  a  neutral  location  without  (the  managed 
care  organization)  personnel  present  might  alleviate  any  response  bias.  Also,  physicians 
are  strong,  take  charge  individuals  to  deal  with.  Perhaps,  a  physician  facilitator  would  be 
better  suited  to  the  challenge  of  keeping  them  in  the  process  at  hand. 

In  closing,  the  process  seemed  to  provide  some  valuable  information,  the 
comments,  reflections  and  suggestions  made  above  are  meant  to  enrich  the  research. 


APPENDIX  P 
MACKINNON'S  TOP  12  DEFINITIONS 


Preventable  drug-related  morbidity 

MacKinnon 

events 

detected 

Faris 

events 

detected 

Outcome: 

Secondary  myocardial  infarction 
Pattern  of  Care: 

1.  History/diagnosis  of  myocardial  infarction 

2.  No  use  of  ASA  and/or  beta  blocker  (e.g., 
metoprolol) 

24 
(15.1%) 

54 
(4.0%) 

Outcome: 

ER  visit/hospitalization  due  to  hyperglycemia 
Pattern  of  Care: 

1 .  Use  of  an  oral  hypoglycemic  agent  (e.g., 
chlorpropamide,  etc.) 

2.  Hemoglobin  Ale  level  not  done  at  least  every 
6  months 

18 
(11.4%) 

N/Aa 

Outcome: 

ER  visit/hospitalization  due  to  hypothyroidism 
Pattern  of  Care: 

1 .  Use  of  thyroid  or  antithyroid  agent  (e.g., 
levothyroxine,  propylthiouricil,  etc.) 

2.  T4/TSH  not  done  before  therapy  starts  and  at 
least  every  1 2  months  thereafter 

12 
(7.6%) 

103 

(7.6%) 

Outcome: 

ER  visit/hospitalization  due  to  hypoglycemia  or 
hyperglycemia 
Pattern  of  Care: 

1.  Use  of  insulin 

2.  Hemoglobin  Ale  level  not  done  at  least  every 
6  months 

10 
(6.3%) 

N/Aa 

Outcome: 

Acute  renal  failure  and/or  renal  insufficiency 
Pattern  of  Care: 

1 .  Use  of  an  ACE  inhibitor 

2.  BUN/serum  creatinine  not  done  at  initiation  of 
therapy  and  at  least  every  3  months  thereafter 

10 
(6.3%) 

1 
(0.1%) 
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Preventable  drug-related  morbidity 

MacKinnon 

events 

detected 

Faris 

events 
detected 

Outcome: 

Gastritis  and/or  upper  GI  bleed  and/or  GI 
perforation  and/or  GI  ulcer  and  anemia 
Pattern  of  Care: 

1 .  NSAID  use  for  at  least  1  month 

2.  No  concurrent  use  of  a  cytoprotective  agent 
(misoprostol) 

3.  Hemoglobin/hematocrit/CBC  not  done  within 
30  days  of  the  start  of  therapy  or  not  done  at 
least  every  three  months  thereafter 

8 
(5.1%) 

N/Aa 

Outcome: 

ER  visit/hospitalization  due  to  congestive  heart 
failure  and/or  fluid  overload 
Pattern  of  Care: 

1 .  History/diagnosis  of  high  blood  pressure  (over 
1 40/90)  and/or  congestive  heart  failure 

2.  NSAID  use  for  at  least  3  months 

8 
(5.1%) 

80 
(5.9%) 

Outcome: 

Acute  urinary  retention 
Pattern  of  Care: 

1 .  History/diagnosis  of  benign  prostatic 
hypertrophy  (BPH) 

2.  Use  of  an  anticholinergic  agent 

7 
(4.4%) 

4 
(0.3%) 

Outcome: 

ER  visit/hospitalization  due  to  extreme 
hypoglycemia 
Pattern  of  Care: 

1 .  History/diagnosis  of  diabetes 

2.  Use  of  beta-adrenergic  blocking  agent  (e.g., 
propranolol,  nadolol,  etc.) 

6 

(3.8%) 

N/Aa 

Outcome: 

ER  visit/hospitalization  due  to  congestive  heart 
failure 
Pattern  of  Care: 

1 .  Diagnosis/history  of  congestive  heart  failure 

2.  Not  on  an  ACE  inhibitor  (e.g.,  captopril, 
enalapril,  etc.) 

6 

(3.8%) 

270 
(19.9%) 
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Preventable  drug-related  morbidity 

MacKinnon 

events 

detected 

Faris 

events 

detected 

Outcome: 

Aminoglycoside  toxicity  (acute  renal  failure  and/or 
renal  insufficiency  and/or  vestibular  damage 
and/or  auditory  damage) 
Pattern  of  Care: 

1 .  Use  of  an  aminoglycoside 

2.  Serum  creatinine  not  done  before  and  after 
therapy  (and  if  therapy  longer  than  7  days,  not 
done  at  least  every  7  days) 

3.  At  least  one  drug  level  not  done 

6 

(3.8%) 

0 
(0.0%) 

Outcome: 

Acute  renal  failure  and/or  renal  insufficiency 
Pattern  of  Care: 

1 .  NSAID  use  for  at  least  3  months 

2.  BUN/serum  creatinine  not  done  at  least  every  3 
months 

6 

(3.8%) 

2 
(0.2%) 

aN/A:  specific  definition  was  not  measured  in  Faris'  population 
NOTE:  MacKinnon  events  from  MacKinnon  (1999) 


APPENDIX  Q 
FARIS'  TOP  10  DEFINITIONS 


Preventable  drug-related  morbidity 

Faris  events 
detected 

MacKinnon 

events 

detected 

Outcome: 

ER  visit/hospitalization  due  to  congestive  heart 
failure 
Pattern  of  Care: 

1 .  Diagnosis/history  of  congestive  heart  failure 

2.  Not  on  an  ACE  inhibitor  (e.g.,  captopril, 
enalapril,  etc.) 

270 
(19.9%) 

6 
(3.8%) 

Outcome: 

ER  visit/hospitalization  due  to  congestive  heart 
failure  and/or  heart  block 
Pattern  of  Care: 

1 .  History/diagnosis  of  congestive  heart  failure 
with  heart  block  or  advanced  bradycardia 

2.  Useofdigoxin 

184 
(13.5%) 

4 
(2.5%) 

Outcome: 

Gastritis  and/or  upper  GI  bleed  and/or  GI 
perforation  and/or  GI  ulcer  and  anemia 
Pattern  of  Care: 

Use  of  2  or  more  NSAIDS  concurrently  for  at  least 

2  weeks 

129 
(9.5%) 

3 
(1.9%) 

Outcome: 

ER  visit/hospitalization  due  to  hypothyroidism 
Pattern  of  Care: 

1 .  Use  of  thyroid  or  antithyroid  agent  (e.g., 
levothyroxine,  propylthiouricil,  etc.) 

2.  T4/TSH  not  done  before  therapy  starts  and  at 
least  every  12  months  thereafter 

103 

(7.6%) 

12 
(7.6%) 

Outcome: 

Asthma  exacerbation  and/or  status  asthmaticus 
and/or  ER  visit/hospitalization  due  to  asthma 
Pattern  of  Care: 

1 .  Diagnosis  of  moderate  to  severe  asthma 

2.  Use  of  a  bronchodilator 

3 .  No  use  of  maintenance  corticosteroid 

89 
(6.5%) 

0 
(0.0%) 
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Preventable  drug-related  morbidity 

Faris  events 
detected 

MacKinnon 

events 

detected 

Outcome: 

ER  visit/hospitalization  due  to  congestive  heart 
failure  and/or  fluid  overload 
Pattern  of  Care: 

1 .  History/diagnosis  of  high  blood  pressure  (over 
140/90)  and/or  congestive  heart  failure 

2.  NSAID  use  for  at  least  3  months 

80 
(5.9%) 

8 
(5.1%) 

Outcome: 

Gastritis  and/or  upper  GI  bleed  and/or  upper  GI 
perforation  and/or  GI  ulcers  and  anemia 
Pattern  of  Care: 

1 .  History /diagnosis  of  ulcers  and/or  GI  bleeding 

2.  NSAID  use  for  at  least  1  month 

63 
(4.6%) 

2 
(1.3%) 

Outcome: 

ER  visit/hospitalization  due  to  depression  and/or 
increase  in  dosage  of  antidepressant. 
Pattern  of  Care: 

1 .  History/diagnosis  of  depression 

2.  Use  of  long-acting  benzodiazepine  (e.g., 
Librium,  Valium,  Azaene/Tranxene,  etc.) 

56 
(4.1%) 

0 
(0.0%) 

Outcome: 

Secondary  myocardial  infarction 
Pattern  of  Care: 

1 .  Diagnosis/history  of  myocardial  infarction 

2.  No  use  of  ASA  and/or  beta  blocker 

54 
(4.0%) 

24 
(15.1%) 

Outcome: 

COPD  exacerbation  and/or  ER  visit/hospitalization 
due  to  COPD 
Pattern  of  Care: 

1 .  Diagnosis/history  of  COPD 

2.  Use  of  a  beta-blocker  (e.g.,  propranolol,  etc.) 

54 
(4.0%) 

2 
(1.3%) 

NOTE:  MacKinnon  events  from  MacKinnon  (1999) 


APPENDIX  R 
PROCESS-TO-OUTCOME  VALUE  AFTER  ADJUSTMENT  FOR  NDPO 


Definition3 

Number  of 
PDRMsb 

Pattern  of  care- 
only 

Outcome  only 

Process-to- 
outcome  value 

48 

89 

0 

173 

1.000 

1 

56 

0 

382 

1.000 

25 

54 

0 

1038 

1.000 

32 

38 

0 

854 

1.000 

7 

35 

0 

426 

1.000 

41 

15 

0 

471 

1.000 

5 

63 

1 

760 

.9844 

33 

184 

9 

653 

.9534 

47 

37 

6 

745 

.8605 

45 

270 

90 

303 

.7500 

36 

54 

33 

63 

.6207 

22 

103 

127 

566 

.4478 

19 

80 

139 

680 

.3653 

16 

19 

53 

157 

.2639 

8 

10 

35 

164 

.2222 

39 

129 

929 

760 

.1219 

20 

44 

365 

318 

.1076 

44 

42 

655 

468 

.0603 

28 

14 

360 

335 

.0374 

aDefinition  number  matches  Appendix  D 

bPDRM:  preventable  drug-related  morbidity 

NOTE:   NDPO  is  the  Number  of  Days  between  the  Pattern  of  care  and  Outcome 
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